DuckLake Solves Streaming to a Lakehouse Without the Small File Nightmare

If you're streaming data to a lakehouse, you've probably hit the small file problem: high-frequency inserts create thousands of tiny files, killing query performance as metadata scanning becomes your bottleneck. DuckLake solves this elegantly with inlining—a novel approach that stores small inserts in your metadata catalog instead of creating files, then flushes them to Parquet when needed. Watch as we: ✅ Set up DuckLake locally in minutes ✅ Insert data and see how inlining prevents tiny files ✅ Query across inlined rows and Parquet files seamlessly ✅ Delete and update data using metadata-only operations ✅ Flush inline data to Parquet manually or with checkpoint This is how DuckLake handles streaming without the complexity of Delta's OPTIMIZE or Iceberg's compaction routines. --- Resources & Links 📄 DuckLake Substack article: https://thefulldatastack.substack.com/p/understanding-ducklakes-inlining 🌐 DuckLake Docs: https://ducklake.select 💾 DuckDB: https://duckdb.org 🏢 MotherDuck (Sponsor): https://motherduck.com – Production DuckDB hosting with analytics APIs