Building Reliable Data Pipelines with Lakeflow on Databricks | Lakeflow Demo
Lakeflow takes you from raw source data to a production-ready, orchestrated pipeline on one platform. Watch Tomasz Bacewicz build a Salesforce pipeline end to end: incremental ingestion with Lakeflow Connect, transformation and data quality checks with Spark Declarative Pipelines generated by Genie Code, no-code data prep in Lakeflow Designer, and orchestration with Lakeflow Jobs. 🎯 What You'll Learn: • How Lakeflow Connect ingests Salesforce data incrementally in a few clicks • How Genie Code generates Spark Declarative Pipelines with built-in data quality checks • How Lakeflow Designer handles last-mile data prep in a no-code interface • How Lakeflow Jobs orchestrates ingestion, transformation, and prep in sequence ⏱️ CHAPTERS: 00:00 - Introduction and platform overview 02:10 - Ingesting Salesforce data with Lakeflow Connect 05:46 - Building pipelines with Genie Code and Spark Declarative Pipelines 11:29 - No-code data prep with Lakeflow Designer 15:03 - Orchestrating everything with Lakeflow Jobs Resources: • Get Started with Data Engineering training (free): https://www.databricks.com/training/catalog/get-started-with-databricks-for-data-engineering-1511 • Lakeflow: https://www.databricks.com/product/data-engineering 🤝 Stay connected: Website: https://databricks.com Twitter: https://x.com/databricks/ LinkedIn: https://www.linkedin.com/company/databricks/posts/ Instagram: https://www.instagram.com/databricksinc/?hl=en Facebook: https://www.facebook.com/databricksinc/ Unify your pipelines and build an agentic data foundation for AI with the Big Book of Data Engineering: https://www.databricks.com/resources/ebook/big-book-of-data-engineering