The Secrets of Deploying AI in Production | Sumti Jairath, Chief Architect at SambaNova
AI agents are moving from demos into production, but many teams run into the same wall: responses that take too long and cost too much. As models grow bigger and reason through more steps before answering, every chained agent adds delay, and that adds up fast at scale. For data and AI professionals, this raises a practical question: how do you pick a model and an inference platform that keeps a multi-agent workflow fast without blowing the budget? And underneath that choice sits a bigger one — how much of the AI stack, from chips to software, should a company actually control? Sumti Jairath is Chief Architect at SambaNova Systems, where he's focused on machine learning, big data analytics, and software-defined hardware since 2017. Before that, he spent close to eight years as a Senior Hardware Architect at Oracle working on hardware acceleration for machine learning and data analytics, and earlier held processor and server design roles at Hewlett Packard and Sun Microsystems. In the episode, Richie and Sumti explore why AI agents are slow and expensive, the architecture behind faster inference, choosing the right model and platform for a task, open-source models and sovereign AI, the full AI infrastructure stack, power efficiency, the AI data center debate, career paths in AI infrastructure, and much more. 0:00 Intro 1:16 Why AI Agents Are Slow And Expensive 3:35 Can AI Be Fast, Cheap, And Good? 9:26 Breaking Down The Full AI Infrastructure Stack 12:27 Choosing Between An API And Hosting Your Own Models 15:49 Why Sovereignty Is Reshaping AI Infrastructure 20:02 The Real Cost Of Scaling Coding Agents 23:57 The Rise Of Personal, Local AI Setups 27:02 Are We Building Too Many AI Data Centers? 34:07 Careers In AI Infrastructure 36:25 What People Are Building With The SambaNova API 40:58 Why AI Tokens Need To Get Radically Cheaper 43:15 Balancing Powerful AI With Efficient AI Find DataFramed on DataCamp https://www.datacamp.com/podcast and on your preferred podcast streaming platform: Apple Podcasts: https://podcasts.apple.com/us/podcast/dataframed/id1336150688 Spotify: https://open.spotify.com/show/02yJXEJAJiQ0Vm2AO9Xj6X Links Mentioned in the Show: • NVIDIA's Groq acquisition, announced around GTC • Chris Olah, Anthropic • Anthropic and OpenAI's coding-agent-driven revenue growth • Ollama • Apple Mac mini • SambaNova careers • Connect with Sumti • AI-Native Course: Intro to AI for Work • Related Episode: From City Sewers to Sovereign AI with Russ Wilcox, CEO at ArtifexAI New to DataCamp? Learn on the go using the DataCamp mobile app Empower your business with world-class data and AI skills with DataCamp for business