AI Coding Agents Write Worse Code Over Time: SlopCodeBench — Gabe Orlanski

Frontier Data Summit 2026: Gabe Orlanski presents SlopCodeBench, a benchmark showing how AI coding agents' code quality changes across repeated edits. Gabe Orlanski presents SlopCodeBench, a coding benchmark that tests AI agents across iterative software development tasks. Agents adapt a codebase as requirements evolve. The talk explains how SlopCodeBench measures redundant code and structural erosion, compares cumulative and isolated performance, and examines how code quality changes across repeated edits. The project received support through Snorkel AI’s Open Benchmarks Grants. IN THIS VIDEO • Why coding benchmarks need to test iteration, not one-shot tasks • How SlopCodeBench measures code verbosity and structural erosion • Cumulative versus isolated coding task performance • Why agents fail to refactor, and whether prompting reduces code slop SPEAKER Gabriel Orlanski, PhD Student, UW-Madison; Project Lead, SlopCodeBench CHAPTERS 00:00 Why coding benchmarks need iteration 01:16 How SlopCodeBench evaluates coding agents 02:06 Measuring code verbosity and structural erosion 03:05 Cumulative versus isolated coding tasks 04:35 Why agents fail to refactor 05:55 Can prompting reduce code slop? 06:24 Why agent performance degrades over time RESOURCES SlopCodeBench paper: https://arxiv.org/abs/2603.24755 Project overview: https://gabeorlanski.github.io/posts/slop-code-bench/ Snorkel AI Open Benchmarks Grants: https://benchmarks.snorkel.ai/