Atoosa Kasirzadeh - Hidden Pitfalls of AI Scientist Agents [Alignment Workshop]

Atoosa Kasirzadeh exposes critical flaws in AI scientist systems that automate research from hypothesis to publication. Her experimental analysis of Agent Laboratory and AI Scientist version two revealed four methodological pitfalls: inappropriate benchmark selection, data leakage, metric misuse, and post-hoc selection bias. Testing confirmed these systems peek at test data during training and systematically choose easier benchmarks while avoiding representative ones. With AI-generated papers already accepted at ACL and ICLR conferences, Kasirzadeh warns that automated scientific discovery risks undermining research integrity without proper validation frameworks. Note: The opinions shared in this event are those of the speaker(s) and may not represent the views of FAR.AI or their affiliated organizations.