Ursa: A Lakehouse-Native Data Streaming Engine for Kafka

During StarRocks Summit 2025, Neng Lu (StreamNative) introduced Ursa, a Kafka-compatible, lakehouse-native streaming engine. 👉 Watch the recording to see how Ursa simplifies real-time ingestion and reduces costs, including: 📌 Lakehouse-Native Streaming — Writing directly to open table formats like Apache Iceberg and Delta Lake on object storage. 📌 Operational Simplicity — Eliminating broker disk storage, leader-based replication, and external connectors. 📌 Unified Ingestion + Analytics — Enabling stream–table duality that shortens time-to-insight and lowers infrastructure costs by up to 10× About StarRocks Summit 2025: This was our first global summit, presented by CelerData — a one-day, free, virtual event that brought together thousands of engineers and practitioners from around the world. The day featured 25+ engineer-led sessions from teams at Coinbase, Intuit, Pinterest, Demandbase, TRM Labs, and more, all sharing real architectures, hard-earned lessons, and what worked in production across real-time analytics, the lakehouse, and AI. ---------------------------------------------------------------------------------------------------------------------- Timestamps 00:00 Intro 00:59 The Evolving Data Landscape 02:06 Streaming to Lakehouse — Challenges 04:59 Design Goals 06:18 Ursa — A Lakehouse-Native Data Streaming Engine 08:28 Components Overview 09:57 Ursa Stream Format 10:23 Ursa Stream Offset Index 11:25 Append Entries to a Ursa Stream 12:58 Read Entries from a Ursa Stream 14:21 Broker Discovery 15:09 Local Write Buffer and Distributed Read Cache 15:25 Compaction Service — Motivation 16:27 Compaction Service — Implementation 17:21 Stream–Table Duality 17:42 Catalog Integration & External Table 18:08 Internal Table 18:30 External Table 19:01 Streaming-Augmented Lakehouse 19:34 Data Streaming Cost Analysis 20:21 Q&A ----------------------------------------------------------------------------------------------------------------------- Learn more at https://celerdata.com/ Connect with us: LinkedIn: https://www.linkedin.com/company/celerdata/ Twitter: https://twitter.com/celerdata CelerData Website: https://celerdata.com/ StarRocks GitHub: https://github.com/StarRocks/StarRocks StarRocks Website: https://www.starrocks.io/ Slack: https://starrocks.io/redirecting-to-slack #DataAnalytics #DataEngineering #RealTimeAnalytics #StarRocksSummit2025 #RealTimeData #OLAP #DataAnalyst #DataEngineer #DataInfrastructure #userfacinganalytics #Database #AnalyticalDatabase