Delta Lake for Rust Hacking: AI Code Ethics, CRC Cache Bugs, & Arrow Forks (Jan 27, 2026)

Join Robert Pack and R. Tyler Croy for a technical sync on the latest hurdles in the delta-rs ecosystem. This session moves from high-level project philosophy—like how to track the provenance of AI-generated contributions—to deep-dive troubleshooting of SQL warehouse caching and write-path optimizations. Key Discussion Points 🔹 AI Code Provenance: Navigating the legal and technical "paper trail" for code generated or assisted by AI within the project. 🔹 The CRC/Log Caching Bug: Investigating why certain SQL warehouses fail when they cache Delta logs that lack CRC (Cyclic Redundancy Check) files. 🔹 Optimizing Writes: Ongoing architectural work to remove eager snapshotting, making write operations leaner and faster. 🔹 Ecosystem Stability: Addressing the friction between rapid development and the need for a reliable production engine. 🔗 GitHub: https://github.com/delta-io/delta-rs 0:00 AI Code Provenance & "License Laundering" (GPL/AGPL implications) 3:11 Delta Log Caching Issue: Missing CRC files in SQL Warehouses 6:52 CDC and CDF: Handling insert, delete, and pre/post-image labeling 9:50 Nullability Assertions: Handling schema inconsistencies in DataFusion 25:00 – Performance: Removing Eager Snapshots & the Table Provider bottleneck 41:00 – Forking the Arrow Engine: Integration with Tokio & String View optimizations