He lied to his AI and told it hacking real banks was 'just a simulation'
He expected to see a CSS file, but he was shocked by his discovery: His AI system stole API keys from a bank while he was asleep. In this episode of AI Cyber Podcast, Confidence Staveley sits down with James Kettle, the Director of Research at PortSwigger, who spent one year building an AI system to discover if AI can conduct novel security research. We discussed the importance of evaluation in AI research, the ethical considerations of using AI in cybersecurity, and the importance of collaboration between AI and human researchers. We also spoke about a new attack concept called shared parser confusion, and the speed at which open-source AI models are catching up to frontier models. What You'll Learn: - James Kettle leads the research team at Portswigger, known for Burp Suite. - He has presented at major cybersecurity conferences for over a decade. - Kettle's research interrogates whether AI can conduct novel security research. - HTTP desync attacks exploit vulnerabilities in the HTTP 1.1 protocol. - The HTTP Terminator system automates the research process using AI. - Evaluation is crucial in AI research to filter out ineffective ideas. - Legal and ethical considerations are paramount in AI-driven research. - Manipulating AI can enhance its effectiveness in research tasks. - The cascade process is essential for driving discoveries in research. - Collaboration between AI and human researchers can lead to significant findings. Connect with Our Guest: James Kettle https://www.linkedin.com/in/james-kettle-albinowax/ ⏱️Timestamps: 00:00 Can AI Perform Novel Security Research? 03:00 The AI Experiment Begins 06:00 When AI Discovered Something Unexpected 09:00 AI Finds API Keys While He Sleeps 12:00 Can AI Really Conduct Novel Research? 15:00 Why Evaluation Matters in AI Research 18:00 The Ethics of AI-Powered Cybersecurity 21:00 Human Researchers vs AI Collaboration 24:00 What Is Shared Parser Confusion? 27:00 A New Approach to Security Attacks 30:00 Open-Source AI vs Frontier Models 33:00 The Future of AI Security Research ˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍ ✅ For Business inquiries ►[email protected] ˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍ ✅ 𝐎𝐭𝐡𝐞𝐫 𝐕𝐢𝐝𝐞𝐨𝐬 𝐘𝐨𝐮 𝐌𝐢𝐠𝐡𝐭 𝐁𝐞 𝐈𝐧𝐭𝐞𝐫𝐞𝐬𝐭𝐞𝐝 𝐈𝐧 𝐖𝐚𝐭𝐜𝐡𝐢𝐧𝐠: 👉 https://www.youtube.com/watch?v=z-vY-r654bY 👉 https://www.youtube.com/watch?v=OWoWWJmlfNs 👉 https://www.youtube.com/watch?v=9EhlSHIBb-4 👉 https://www.youtube.com/watch?v=JoMumubmpBg ˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍ 𝐃𝐢𝐬𝐜𝐥𝐚𝐢𝐦𝐞𝐫: Copyright Disclaimer Under section 107 of the Copyright Act of 1976, allowance is made for "fair use” for purposes such as criticism, comment, news reporting, teaching, scholarship, education, and research. Fair use is a use permitted by copyright statute that might otherwise be infringing. ˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍ 🔎 𝐑𝐞𝐥𝐚𝐭𝐞𝐝 𝐏𝐡𝐫𝐚𝐬𝐞𝐬: AI security research,AI cybersecurity,novel security research,AI hacking,autonomous security research,James Kettle,PortSwigger,AI Cyber Podcast,Confidence Staveley,AI vulnerability research,cybersecurity AI,AI agents cybersecurity,ethical AI hacking,security testing AI,open source AI models,frontier AI models,AI security testing,human AI collaboration,cybersecurity research,Can AI Perform Novel Research? ˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍ #️⃣𝐇𝐀𝐒𝐇𝐓𝐀𝐆: #aicybersecurity #aisecurity #Cybersecurity #AIResearch #EthicalHacking #JamesKettle #PortSwigger #AIHacking #SecurityResearch #ai ˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍˍ ►Like This Video. ►Share this Video with your friends. ►Subscribe to the channel if you haven't already.