Creating a Digital Twin with Spark Real-Time Mode and Lakebase

Watch how Spark Structured Streaming Real-Time Mode, Lakebase, a Databricks App, and Unity Catalog work together to help manufacturing teams detect a machine component running hot, hold production, and investigate the event before a batch is compromised. In this demo, you’ll follow a temperature reading from a machine component through the message bus, Spark Structured Streaming Real-Time Mode, Lakebase, and Unity Catalog managed tables. See how the digital twin updates as each reading arrives, how the app triggers an inspection and holds and resumes production, and how Genie helps teams query what happened from governed Unity Catalog tables. 🎯What you’ll learn Detect operating-limit breaches with Spark Structured Streaming Real-Time Mode Store live asset state in Lakebase and stream updates to a Databricks App with Server-Sent Events Preserve device time and catch duplicate or late readings Move Lakebase changes into Unity Catalog managed tables for history, BI, predictive maintenance, and Genie Hold production, call an inspection, and resume the line from the Databricks App ⏱️ CHAPTERS: 00:00: Introduction and the plant floor problem 02:40: Intervention vs. delay (reactive vs. real-time) 03:36: Requirements for a useful digital twin 04:32: Databricks stack for a real-time digital twin 08:13: Live demo: CNC machine digital twin app 13:02: Summary & value to the plant 📖 RESOURCES Learn more about Real-Time Mode in Apache Spark: https://www.databricks.com/product/data-engineering/spark-real-time-mode Learn more about Lakebase Change Data Feed: https://docs.databricks.com/aws/en/oltp/projects/lakebase-cdf Learn more about LakeTS: https://databricks-solutions.github.io/lakets/ 🤝 STAY CONNECTED: Website: https://databricks.com Twitter: https://x.com/databricks/ LinkedIn: https://www.linkedin.com/company/databricks/home/ Facebook: https://www.facebook.com/databricksinc/