Extract Podcast Ep 11 - AI Agents, Custom Tools and the Future of Web Scraping

AI agents are changing how we build software, access the web and automate everyday work. In Episode 11, Ayan and John explore the latest developments in AI coding tools, custom harnesses, web scraping and agent security. They discuss the rise of modular AI workflows, the limitations of prompt-to-data systems, why deterministic code still matters, and how teams can evaluate and secure agents before putting them into production. John also shares new Scrapy projects, including a lightweight CDP integration and an opinionated project template. The conversation covers Linux and open-source development, model pricing, agent SDKs, proactive desktop agents and the future of web access. Plus, they wrap up with a few thoughts on French food and the perfect croissant. Useful links: Zyte: https://www.zyte.com/ Zyte Open Source: https://www.zyte.com/open-source/ The State of Web Access report: https://www.zyte.com/whitepaper-ebook/ Zyte CDP documentation: https://docs.zyte.com/zyte-api/usage/cdp.html Chapters: 00:00 Intro: AI models and hidden launches 02:54 Custom harnesses, tools and developer workflows 11:05 The State of Web Access report 15:23 Can prompt-to-data replace traditional scraping? 18:09 Evaluating AI systems and production reliability 20:40 Securing agents against prompt injection 23:07 Scrapy CDP and browser automation 26:15 Linux, i3 and the open-source ecosystem 30:18 AI model pricing and performance 33:55 Agent SDKs, LangGraph and model-agnostic workflows 37:04 The rise of proactive desktop agents 38:59 Food, croissants and episode wrap-up Subscribe for more conversations about AI, web scraping, open source and developer tools, and let us know what you think in the comments.