AI Cybersecurity: When an AI Agent Cheats on a Hacking Exam — Dawn Song, UC Berkeley

Frontier Data Summit 2026: Dawn Song on what happens when an AI agent cheats on a hacking exam in the real world, and how to build benchmarks that can't be gamed. Alex Ratner sits down with Dawn Song of UC Berkeley to discuss Agents' Last Exam, a benchmark of long-horizon, economically valuable tasks across more than 50 sectors, and her lab's cybersecurity benchmarks, CyberGym and ExploitGym. They cover the OpenAI–Hugging Face incident, why benchmark builders need an adversarial mindset, how reward hacking turns into misalignment in training, the offense-defense asymmetry in AI cyber capabilities, and where human expertise still matters most. IN THIS VIDEO • Why Agents' Last Exam targets long-horizon, real-world work across 50+ sectors • What CyberGym and ExploitGym reveal about fast-rising AI cyber capabilities • The OpenAI–Hugging Face incident, where an agent solving ExploitGym tasks cheated in the real world • Why benchmark builders need an adversarial mindset: isolation, untrusted inputs, and prompt injection • How benchmark hacking becomes a training and alignment problem, and how BenchJack red-teams evals automatically • Why attackers have the near-term edge in AI cyber, and what verifiable code generation could change SPEAKERS Dawn Song, Professor, UC Berkeley; Director, Berkeley RDI; VP of AI Research, Meta Alex Ratner, CEO & Co-Founder, Snorkel AI CHAPTERS 00:00 Agents' Last Exam: why the field needed it 03:21 Closing the gap: longer horizons and broader coverage 07:00 CyberGym and ExploitGym 11:40 The OpenAI–Hugging Face incident 14:18 Building secure benchmarks and environments 19:20 Benchmark hacking as a training and alignment problem 21:51 Benchmark red teaming and BenchJack 23:25 Offense, defense, and the cyber "perfect storm" 28:39 Where human expertise matters most 30:21 Practical advice for secure benchmarks 33:49 Open benchmarks and continuous red teaming 34:44 The future of human-AI collaboration evals RESOURCES Agents' Last Exam: https://rdi.berkeley.edu/blog/agents-last-exam CyberGym: https://rdi.berkeley.edu/blog/cybergym BenchJack paper: https://arxiv.org/abs/2605.12673 Snorkel AI Open Benchmarks Grants: https://benchmarks.snorkel.ai/