Dask Futures for General Parallelism
Dask lets you parallelize your Python code, regardless of what libraries you use. In this example we show Dask futures, a low level API to run your Python functions in parallel either on your own computer or on a cluster. If you're new to Dask, this is a great place to start. We start with just a single machine, leveraging all of the cores on a macbook, and then at the end scale out to a cluster. If you'd like to try this demo yourself, the notebook lives here: https://github.com/coiled/examples Get started with Coiled for free: https://coiled.io/start Key Moments 00:00 Intro 01:43 Parallelize simple for loop 03:40 Complex workloads 06:18 Scale with Coiled 09:22 Exit and next steps