How AI Agents Work: Inside Agentic Inference

What happens when an AI agent starts working, and how do we measure its performance? In this episode of Inside the Architecture, we break down how AI agents use models, tools, and context to complete tasks. Follow the work through animated diagrams, then explore how benchmarks and inference infrastructure shape the experience. You’ll learn: How agentic workflows differ from a single prompt and response How tool calls, subagents, and growing context change the workload Why fixed-length tests and recorded-session replay measure different things How to interpret interactivity, latency, concurrency, and throughput Where NVLink, NVIDIA Dynamo, NVFP4, and TensorRT-LLM fit into the system No prior experience with inference infrastructure required. #AgenticAI #nvidia Benchmark results and methodology: https://developer.nvidia.com/blog/nvidia-vera-rubin-and-blackwell-set-a-new-standard-for-agentic-ai-performance-per-watt/ NVIDIA Dynamo architecture: https://developer.nvidia.com/blog/introducing-nvidia-dynamo-a-low-latency-distributed-inference-framework-for-scaling-reasoning-ai-models/ 00:00 From Chat to AI Agents 00:48 How AI Agents Work 02:28 How Agentic Benchmarks Work 03:37 Which Metrics Matter? 04:32 Where Do Performance Gains Come From? 05:22 Understanding Benchmark Results 06:35 The System Behind the Agent 06:52 Where to Learn More