Why AI Safety Evaluations Are Broken: The Test-Time Compute Problem | Noam Brown (OpenAI)

Noam Brown (OpenAI) on why AI evaluations, including safety evaluations, mislead when they ignore how much a model thinks at test time. Brown shows how a single bar chart can misrepresent a model's capability: plot performance against output tokens, cost, or time, and gains that looked flat become large, because reasoning models keep improving with test-time compute. For reasoning and agentic tasks, how long a model thinks is a hidden variable in most current benchmarks. The safety consequence is direct: a model evaluated at a low inference budget can look harmless, while an actor spending far more on inference reaches more dangerous capability. His recommendations: publish benchmarks with a compute, cost, or time axis, track inference usage on leaderboards, and write test-time compute into preparedness frameworks and responsible scaling policies when setting safety thresholds. Chapters 0:00 GPT-5.5 vs 5.4: why benchmarks miss the story 0:26 Plotting performance against tokens 0:48 Models don't plateau: they scale with compute 1:35 Evaluate as a function of compute, not a single bar 2:20 Better benchmarks: ARC-AGI's cost curves 2:38 Why safety evaluations are broken 2:45 How low-budget evals hide catastrophic risks 3:39 Three concrete recommendations 4:24 Preparedness frameworks and test-time compute More AI safety research: https://far.ai Alignment Workshop playlist: https://youtube.com/playlist?list=PLBY5kyt_LfFg&si=DA_ufycPrJV74Pam FAR.AI is a research nonprofit working to ensure the safe development of advanced AI. We host the Alignment Workshop series and publish frontier alignment research.