How voice AI agents control your browser

Code for this episode → https://g.dev/cloud/repositories Gemini Live API docs → https://g.dev/cloud/gemini-live CDP Page commands → https://g.dev/cloud/devtools-protocol How does an AI voice agent control a real browser while keeping the conversation going? This episode explains a design pattern for live voice agents: run suitable browser actions in the background, then use the next screenshot to observe the result. Keep read operations synchronous when the model needs their returned data to decide what happens next. We cover the screenshot, reason, act loop, Chrome DevTools Protocol, tool classification through a callback, and server-side confirmation for consequential actions. Chapters: 00:00 - Voice-Controlled Browser Demo 00:12 - Giving a Live Voice Agent Browser Control 00:46 - The Browser Control Loop 01:27 - From Model Decisions to Clicks: Chrome DevTools Protocol 01:57 - Why This Agent Uses Screenshots 02:43 - Keeping Slow Tools from Stopping the Conversation 03:45 - Async Actions vs. Synchronous Reads 04:11 - Classifying Tools with a Callback 04:36 - Human Confirmation and Server-Side Guardrails 05:25 - Next Episode: Long-Term Agent Memory More resources: CDP Input commands → https://g.dev/cloud/page-devtools-protocol CDP Page commands → https://g.dev/cloud/input-devtools-protocol ADK tool callbacks → https://g.dev/cloud/types-of-callbacks Follow Annie online: LinkedIn → https://goo.gle/annie-linkedin X → https://goo.gle/annie-x Watch more The Omni App → https://g.dev/cloud/modern-ai-app 🔔 Subscribe to Google Cloud Tech → https://goo.gle/GoogleCloudTech #AIAgents #BrowserAutomation #ToolCalling #MultimodalAI Speaker: Annie Wang Products Mentioned: Gemini Live API, Agent Development Kit, Chrome