Dask DataFrame is Fast Now

In this webinar, Patrick Höfler and Rick Zamora show how recent development efforts have driven performance improvements in Dask DataFrame. Key Moments 00:00 Intro 00:19 Dask DataFrame is fast now 02:06 Historical pain points 03:51 PyArrow-backed strings in Dask 06:04 Demo: PyArrow strings 08:53 Demo: Task-based shuffling is slow 11:11 Better performance with P2P shuffling 16:29 Sub-optimal queries 21:28 How we made Dask 100x faster 26:03 Demo: P2P shuffling and query optimization 27:30 TPC-H benchmarks 32:37 Summary of benchmark results 34:52 GPU-accelerated Dask DataFrames with RAPIDS 41:27 Q&A ---- Additional resources: - TPC-H benchmark results https://tpch.coiled.io/ - Query optimization with Dask DataFrame https://github.com/dask-contrib/dask-expr