Scientist AI: A Safe-by-Design Alternative to Agentic AI | Iulian Serban (LawZero)

Iulian Serban (LawZero) on Scientist AI: a safe-by-design alternative to agentic AI, built to reason without goals or agency. Serban, Senior Director at LawZero, argues that patching today's LLMs will not make them safe, and proposes a different design: an AI modeled on an idealized scientist that forms and evaluates hypotheses without ego, goals, or agency. His case rests on three ideas: separating intelligence from agency, building falsifiable hypotheses through an "explainer" and a "predictor," and separating facts from opinions. He points to a recent result in which the system grows more honest as it grows more capable, weakening the usual link between capability and danger. Chapters 0:00 What is LawZero & Scientist AI? 0:32 The risks: from sycophancy to scheming 0:48 Why patching today's models isn't enough 1:01 Scientist AI: safe by design 1:41 Separating intelligence from agency 2:31 Falsifiable hypotheses: explainer and predictor 3:30 Separating facts from opinions 4:02 Architecture and consequence invariance 4:37 Summary: auditable, verifiable, disinterested More AI safety research: https://far.ai Alignment Workshop playlist: https://youtube.com/playlist?list=PLBY5kyt_LfFg&si=ec7-wvJVb3aK0puU