The Hidden Cost of Batch Processing | Modern Data Engineering Pipelines

Confluent Developer
1,145 views •

https://confluent.io | How a 59-minute delay can cost millions and destroy customer trust. What happens when your data pipeline is perfectly functional but fundamentally flawed? This video explores the hidden costs of batch processing through the story of an online retailer whose "next-day delivery" promise becomes a liability when inventory systems fall 59 minutes behind reality. From the Taylor Swift Eras Tour Ticketmaster disaster to everyday e-commerce failures, learn why businesses like Walmart are abandoning batch processing for real-time event streaming. Discover how "Shift-Left" analytics is transforming data engineering from scheduled batch jobs to continuous stream processing. Technologies covered: Stream processing, Event-driven architecture, Apache Kafka®, Apache Flink®, Shift-Left Analytics, Real-time data pipelines 📚 CHAPTERS: LEARN MORE ► Confluent Developer: https://developer.confluent.io CONNECT Subscribe, if you dare: https://www.youtube.com/@ConfluentDeveloper?sub_confirmation=1 Community Slack: https://confluentcommunity.slack.com X: https://x.com/confluentinc Linkedin: https://www.linkedin.com/company/confluent GitHub: https://github.com/confluentinc Site: https://developer.confluent.io ABOUT CONFLUENT DEVELOPER Confluent Developer provides comprehensive resources for developers looking to learn about Apache Kafka®, Apache Flink®, Confluent Cloud, Confluent Platform, and any other technology related to the broader Data Streaming Platform. Content on Confluent Developer includes courses, getting started guides, topical deep-dives, patterns, tutorials, and listings of community events. Learn more at https://developer.confluent.io. #apachekafka #apacheflink #confluent

Keyboard shortcuts

On. Switch them off if they collide with your assistive tools; ? still opens this sheet.

Go to

Press g then the letter.

  • gh Latest
  • gs Sources
  • gm Media
  • gv Videos
  • gp Podcasts
  • gc Calendar
  • gd Decoder
  • gz Dataviz
  • ga Datasets
  • gb Blog
  • gk Markets
  • gj Careers
  • gn Prompt Notebook

On this page

  • / Focus search, where there is one
  • t Back to top
  • ? This list
  • Esc Close