Data engineering
Pipelines, lakehouses, warehouses, streaming and the platforms that move data
Latest stories See all →
- Still Batching Streaming Data into Files? Send Kafka Straight to the Lake with Kafka Connect [OSS Tables Deep Dive]
- What The Modern Data Report’s 5 Emerging Trends Mean for the Public Sector
- Data Engineering SQL Patterns Part 2: Running Totals
- PostgreSQL JSONB: The Feature That Can Save You From Creating 20 Tables
- Your DuckDB Flock Deserves A Mother: MotherDuck Now Offers Open Source DuckDB Support
- 2026/09/14 - Kaitlyn HenryYour DuckDB Flock Deserves A Mother: MotherDuck Now Offers Open Source DuckDB SupportWe're launching commercial support for open-source DuckDB, so every organization can get the power of a modern data engine, with direct access to database experts.
- AIBuilding a Scalable Lakehouse with Apache Iceberg and Snowflake Horizon Catalog
- Third-Party Risk Assessments | How Confluent Helps You Move Faster with Confidence
- Aiven, Confluent, Redpanda, StreamNative and Ververica Form Streamhouse Working Group
- How Brands Are Rethinking Customer Data, and What It Means for Marketers
- Snowflake DCM Projects Setup Guide: Database Change Management
- Databricks on AWS, Built From Scratch: What the Platform Gets Right
- Streamhouse Is Bigger Than Us: Why We're Building the Next Architecture Together
- Obsidian Security Named a Databricks Silver Partner — One of Only Two Security Vendors at Silver or Above
- Apache Iceberg Python 0.12.0 Release
- 9½ Core Skills Every Data Engineer Needs
- Databricks Now Has a Governed Semantic Layer. Should It Be Yours?
- Databricks Now Has a Governed Semantic Layer. Should It Be Yours?
- Coalesce 2.0: The Agentic Data Engineering Platform
- Designing a Shared Platform to Serve Competing Institutional Clients Without Cross-Contamination
- Sync Your Ometria Contacts Anywhere: Announcing the Integrate.io Ometria Connector
- 2026/09/15 - Dumky de WildeDatabase 101: when to choose which databaseSQLite, Postgres, DuckDB and 20 others: they all write, store and read data. The questions you ask every day tell you which one you need.
- Scaling ML Model Inference through architectural separation
- Apache Flink Kubernetes Operator 1.16.0 Release Announcement
Videos
- Ingest data from 600+ sources into BigQuery with automated schema management
- Databricks for Good x Toast91: Redefining how Young People access Mental Health Support
- Databricks for Good x Global Orphan (GO) Project: Turning Insight into Impact
- How to Build and Serve Production ML Features | Databricks Feature Store Demo
- We rank #1 on Snowflake's MADQA Leaderboard! But what does it really mean to be #1?
- British Red Cross Modernizes Emergency Response with Databricks
- Queues with Postgres - Goodbye SQS, RabbitMQ and Kafka
- The Edge: The Intelligence Edge (with Databricks)
- Databricks Genie Data Democratization at Amadeus | Data + AI Executive Series: Fast 5
- How to Connect Your Coding Agent and IDE to Databricks Compute | Databricks SSH Tunnel Demo
- How Zeta Global Powers AI Agents With Snowflake AI Data Cloud
- How Babylist Reclaimed Engineering Time on Snowflake's AI Data Cloud
Podcasts
- Talk Python To Me
- The Data Engineering Podcast
- MLOps Community
- Software Engineering Daily
- Data Engineering Weekly
- Data Mesh Radio
- AI Engineering Podcast
- The Python Podcast.__init__
- Data Engineering Central Podcast
- The Data Flowcast
- The Engineering Side of Data
- Data Brew by Databricks
Recent episodes
- The Future of Data Engineering in the Age of AI | Erfan Hesami
- Why Enterprise AI Agents Fail Beyond the Model With Databricks
- What Context Really Means in Data Engineering and AI
- Treat your data pipelines like code, because your agents already do | MotherDuck's Jordan Tigani
- #562: DuckLake: The Lakehouse That's Just SQL and Parquet
- Why AI Agents Will Break Modern Data Engineering (And How to Fix It) w/ Christophe Blefari (Nao)
- Building self-healing Airflow pipelines at ATC Drivetrain
- The Rise of Organized Retail Crime at Big Box Stores
- Big Tech Laid Him Off. He Realized He Didn’t Need a Job. — Asian Dad Energy
- The Return of Conceptual Data Modeling w/ Sami Hero (CEO of Ellie.ai)
- NVIDIA's $12.93B Hugging Face Deal, Anthropic's Compute Buildout, and a Blowout AI Earnings Week
- #376 Rethinking the Data Stack in the age of AI with Tristan Handy, President of Fivetran + dbt Labs