Databricks Pipelines: Write to One Unity Catalog Table from Multiple Sources with FLOWs

Databricks
1,185 views •

Learn how to write to a single managed Unity Catalog table from multiple data sources using flows in Databricks pipelines. WHAT YOU'LL LEARN • Single Table, Multiple Sources: Write from multiple independent streams into one target table within the same pipeline • Independent Offsets: Track progress per flow with stable flow-name checkpoints • Zero-Reprocessing Migration: Migrate existing Structured Streaming jobs seamlessly using import_checkpoint in Python TIMESTAMPS 0:00 - One table, multiple sources, one pipeline 0:11 - Create the managed target (SQL & Python) 0:17 - Add one flow per source 0:28 - Stable flow names & checkpoints 0:35 - Migrate Structured Streaming (import_checkpoint) 0:50 - Checkpoint copying & resume 0:59 - Complete SQL syntax example

Keyboard shortcuts

On. Switch them off if they collide with your assistive tools; ? still opens this sheet.

Go to

Press g then the letter.

  • gh Latest
  • gs Sources
  • gm Media
  • gv Videos
  • gp Podcasts
  • gc Calendar
  • gd Decoder
  • gz Dataviz
  • ga Datasets
  • gb Blog
  • gk Markets
  • gj Careers
  • gn Prompt Notebook

On this page

  • / Focus search, where there is one
  • t Back to top
  • ? This list
  • Esc Close