Building an Open, Governed Lakehouse with Apache Polaris

We had the pleasure of hosting Yufei Gu (Snowflake) — Apache Polaris PPMC, Iceberg PMC, and Hadoop PMC — for a deep dive into why traditional Iceberg catalogs fall short and how Apache Polaris introduces a truly unified approach to governance. ---------------------------------------------------------------------------------------------------------------------- Timestamps 00:00 Intro and Agenda 02:01 Why We Need a REST Catalog 05:13 Traditional Iceberg Catalog and Performance Concerns 08:16 Iceberg Optimistic Concurrency 10:13 What We Need in a Catalog 11:39 Iceberg REST Catalog 12:10 How Moving Responsibilities From the Client to the Server Fundamentally Changes the Way Iceberg Tables Are Used 15:06 Recent Features and Future Direction 17:13 Apache Polaris (Incubating)   17:40 Cross-Engine Read and Write   18:48 Centralized Security and Access   19:58 Run Anywhere, No Lock-In   20:11 Entity Hierarchy   21:01 Permission Model   22:32 New Feature: Catalog Federation   25:13 New Data Governance Feature: Policy & FGAC   26:48 Community-Driven Roadmap 30:16 Join the Community 31:22 Ron & Yufei Q&A   31:35 What are the one or two core reasons governance is so difficult today in lakehouse environments?   35:19 When evaluating a potential integration, what signals tell you that a tool will fit well within the Polaris ecosystem?   38:27 With the StarRocks–Polaris integration, what stands out to you about StarRocks’ approach to catalog control and integration?   40:55 Do you see Polaris becoming the de facto metadata plane for the lakehouse ecosystem? Where do you see Polaris in the landscape?   43:13 For organizations starting to build their first open data lakehouse—especially those coming from fragmented data systems—what advice would you give them at the very beginning?   46:56 What are you most excited about in the upcoming roadmap, and what do you think users are most looking forward to? 49:48 Questions From Audience   50:00 How are UDF functions defined, and how is backward or forward compatibility with the client enforced in Polaris?   52:55 Does Polaris have integration with other standard catalogs like Glue or HMS? ---------------------------------------------------------------------------------------------------------------------- Learn more at https://celerdata.com/ Connect with us: LinkedIn: https://www.linkedin.com/company/celerdata/ Twitter: https://twitter.com/celerdata CelerData Website: https://celerdata.com/ StarRocks GitHub: https://github.com/StarRocks/StarRocks StarRocks Website: https://www.starrocks.io/ Slack: https://starrocks.io/redirecting-to-slack #DataAnalytics #ApachePolaris #DataEngineering #RealTimeAnalytics #RealTimeData #OLAP #DataAnalyst #DataEngineer #DataInfrastructure #databaseprogramming #apacheiceberg #datalake #datalakehouse