Thunder on multiple cores #342

Activity

1. jwittenbach commented on Jun 22, 2016

   

   jwittenbach

   on Jun 22, 2016

   Contributor

   More actions

   @RijadAlisic Are you trying to using "local mode" or are you trying to run against Spark?
2. freeman-lab commented on Jun 22, 2016

   

   freeman-lab

   on Jun 22, 2016

   Member

   More actions

   Assuming it's local, you can technically run Spark on a single machine, and leverage multiple cores, but running Spark locally also incurs a lot of overhead compared to the generic local numpy use, so I'm not sure it'd be worth it.

   Looking ahead, we want to make dask an optional backend engine for thunder, alongside the current spark and numpy backends, and I think that will give a large boost to local multi-core performance.