delta-explain: Making Delta Lake File Pruning Visible, and Testable in CI

For scan-heavy lakehouse workloads, one of the biggest performance and cost levers is decided before any engine runs: whether a file can be eliminated before it is ever opened. That decision lives in the table's physical layout and metadata, and it is normally invisible. A query can look perfectly reasonable while the layout has already made it expensive. Every file you don't skip is bytes you scan and money you pay. This session introduces delta-explain, a small open-source CLI that makes Delta Lake file pruning visible, measurable, and assertable. Built on delta-kernel-rs, it doesn't reimplement pruning: the kernel computes the surviving file set correctly, and delta-explain attributes the result to its two distinct mechanisms (partition pruning and data skipping) and explains it per file, all from transaction-log metadata, without executing the query in any engine. We'll run it live on the same logical query over two physical layouts, watch the pruning collapse, then turn the same analysis into a CI contract: a threshold and an exit code that fails a pipeline before a regression reaches production and the bill. Speaker: Christian Del Monte, Senior Software Architect at adesso SE Host: Robert Pack, Developer Advocate at Databricks