Qualcomm's AI250 Attacks the AI Inference Memory Bottleneck | Durga Malladi Interview

AI's memory bottleneck is breaking. Qualcomm is directly attacking the Total Cost of Ownership (TCO) for AI inference with its new AI200 and AI250 data center solutions, featuring a projected 10x leap in effective memory bandwidth. I sat down with Durga Malladi, SVP & GM of Technology Planning, Edge Solutions, and Data Center at Qualcomm, for a deep dive into this new hardware. We discuss how their "Near-Memory Computing" architecture in the AI250 is an assault on the token generation bottleneck, specifically targeting the 'decode' phase to boost tokens-per-second and crush TCO. This isn't just about raw performance; it's a strategic "mix and match" play. Durga explains how hyperscalers & CSPs can integrate the AI250's capabilities alongside their own custom silicon, offering a new level of flexibility. We also cover how Qualcomm is leveraging its decades of experience with the Hexagon NPU (from phones to auto) to create these new, highly scaled, direct liquid-cooled rack-level solutions. 🔔 Subscribe to our channel for more great content just like this: https://youtube.com/twimlai?sub_confirmation=1 📖 CHAPTERS =============================== 00:00 - Introduction to Qualcomm's New AI Inference Products 00:33 - AI200 - Direct Liquid Cooled Rack 01:23 - AI250 - Innovative Memory Architecture 02:26 - Target Customers: Hyperscalers and CSPs 03:08 - Lessons Learned Since the Original AI 100 04:40 - Hexagon NPU & Continuity of Architectures Across Devices 05:58 - Qualcomm's Edge in a Competitive Market 07:50 - Deep Dive: Near-Memory Computing Architecture 09:57 - How Architecture Impacts Inference Performance 10:45 - Software and Ecosystem Strategy 12:30 - Go-to-Market Strategy for Data Center AI 13:45 - The Product Roadmap and Annual Cadence 14:38 - Conclusion 🤝 ABOUT OUR GUEST =============================== Durga Malladi is Senior Vice President and General Manager of Technology Planning, Edge Solutions, and Data Center at Qualcomm Technologies, Inc. 🔗 LINKS & RESOURCES =============================== Qualcomm Announces AI200 & AI250: https://www.qualcomm.com/news/releases/2025/10/qualcomm-unveils-ai200-and-ai250-redefining-rack-scale-data-cent Qualcomm Data Center Offerings: https://www.qualcomm.com/artificial-intelligence/data-center Speculative Decoding & Efficient LLM Inference: https://twimlai.com/podcast/twimlai/speculative-decoding-and-efficient-llm-inference/ 🗣️ CONNECT WITH US! =============================== Subscribe to the TWIML AI Podcast: https://twimlai.com/podcast/twimlai/ Follow us on Twitter: https://x.com/twimlai Follow us on LinkedIn: https://linkedin.com/in/twimlai Join our Slack Community: https://twimlai.com/community/ Subscribe to our newsletter: https://twimlai.com/newsletter/ Want to get in touch? Send us a message: https://twimlai.com/contact/ 📸 Camera: https://amzn.to/3TQ3zsg 🎙️Microphone: https://amzn.to/3t5zXeV 🚦Lights: https://amzn.to/3TQlX49 🎛️ Audio Interface: https://amzn.to/3TVFAIq 🎚️ Stream Deck: https://amzn.to/3zzm7F5