Terrorist Use Of AI: Do Current Safeguards Work? | Panel at the United Nations

Adam Gleave (FAR.AI), Adam Hadley (Tech Against Terrorism) and Paul Ash (Christchurch Call Foundation) on whether current AI safeguards hold against terrorist misuse. A panel at a FAR.AI convened discussion at the United Nations during UNGA 81, closing a session that also featured Steven Siqueira (UNOCT), David Scharia (UN CTED) and Brian Tse (Concordia AI). Three organizations that have each built or are building a safety benchmark compare what they are finding. Gleave opens on FAR.AI's AI Security Leaderboard, launched in July, which ranks models by how many universal jailbreaks have been found against them. Roughly half the major models on the market meet the minimal standard it sets, which he takes as evidence the problem is one of implementation rather than open research. Two did not: Gemini and Grok, where hundreds of universal jailbreaks were found at under $300 each, within reach of a low-resource actor. Grok has since moved up the ranking. Ash sets out three concerns driving the Christchurch Call Foundation's work: misuse of AI to build capability, misuse to influence people at scale through synthetic content, and the one he considers most consequential long term, what happens psychologically when someone with a grievance talks to a chatbot or companion AI and finds their reasoning reflected back. The Foundation is weeks away from releasing a multi-turn safety benchmark for chatbots and companion AI, built with a panel of psychology and social science experts. Hadley argues the field should not lose sight of unglamorous threats that already work, noting the volume of terrorist material still circulating online and that ISIS maintains dozens of its own websites. His central concern is obliterated models, which he defines plainly as models where someone has deliberately stripped out all or most of the guardrails. His argument is that while the leading providers, American and Chinese alike, have invested real effort in guardrails, freely downloadable obliterated models undercut all of it, and that this belongs at the top of the agenda rather than off it. The discussion covers why the ecosystem cannot agree on threat models, and the mosaic problem that follows when every model has good safeguards aimed at different things. It covers the gap between alarming benchmark scores and measured real-world uplift, and why Gleave nonetheless thinks uncertainty argues for caution rather than against it. It covers how misuse risk and loss of control risk merge once autonomous agents are in play, and lessons from the cyber and social media eras that the panel thinks are being repeated. On what to do, the recommendations are concrete: better visibility into what models actually do and what threat actors are actually doing, declassified intelligence sharing, multistakeholder structures with civil society and affected communities inside them, proprietary developers implementing safeguards that already exist, open-weight developers experimenting with filtering the most dangerous material out of training data before training rather than patching afterwards, and export control regimes that were not designed with AI in mind being revisited. Audience questions cover whether governments should mandate disclosure from model providers, and whether raising alarm about AI risk could itself radicalize people against the industry. Chapters 0:00 Introducing the panel 2:05 Why this panel, and the benchmarks behind it 2:58 Adam Gleave: the AI Security Leaderboard 7:03 Paul Ash: three things troubling the Christchurch Call Foundation 10:46 Adam Hadley: the unglamorous threats we should not forget 15:34 Defining an obliterated model 16:27 What keeps you up at night 16:56 How misuse risk and loss of control risk merge 18:30 Capability times intent, and what AI changes 19:54 Testing chatbots and companion AI for harm 23:22 Why the ecosystem cannot agree on threat models 27:55 Lessons from the history of cyber 28:27 Getting policymakers to care before an attack 33:43 Closing recommendations 37:36 What developers should do, including pre-training filtering 39:23 Two requests for government 41:36 Q&A: mandatory disclosure or voluntary sharing? 48:13 Q&A: radicalization against the AI industry itself 55:51 Closing Adam Gleave (FAR.AI), Adam Hadley (Tech Against Terrorism) and Paul Ash (Christchurch Call Foundation) agree that the leading models' guardrails have improved, and that freely available models with those guardrails stripped out undermine the whole effort unless the field treats it as a priority. More AI safety research: https://far.ai FAR.AI is a research nonprofit working to ensure the safe development of advanced AI. We host the Alignment Workshop series and publish frontier alignment research.