Measuring AI R&D Automation: What Happens When AI Automates AI Research | Alan Chan (GovAI)

Alan Chan (GovAI) on AI R&D automation: why AI systems automating AI research matters, and what technical researchers should measure to track it. Chan sets out the high-level case. AI companies are investing heavily in automating AI R&D, with CEOs at OpenAI, Anthropic, and Google DeepMind giving timelines around 2028, and coding agents already doing a large share of the software engineering involved. The evidence of progress is both quantitative, including the METR time-horizon trend where models now complete tasks that take human experts more than 16 hours, and qualitative, with some researchers orchestrating teams of agents rather than coding, and models matching human experts at predicting which research ideas will work. He then separates two consequences. Faster AI progress could deliver high-impact capabilities, medical and otherwise, well ahead of expectations, while also compressing the time available to prepare for dual-use ones. Human oversight could weaken through a knowledge gap, as people lose ground-level understanding of what is happening in R&D, and through concentration of decision-making, as the independent viewpoints that improve research decisions drop out. Chan is clear that the net effects are uncertain, and closes with research the field could do: experiments isolating what actually drives AI progress, metrics for oversight quality, and more evaluations of AI R&D, including for safety R&D specifically, so differential progress can be tracked. Chapters 0:00 Measuring AI R&D automation (GovAI) 0:31 AI companies are betting on automating AI R&D 1:01 Evidence: the METR time-horizon trend 1:54 Qualitative change: orchestrating agents, research taste 2:40 How far we have come in two years 3:14 Implication 1: faster AI progress 4:02 High-impact and dual-use capabilities arriving sooner 4:54 Implication 2: weaker human oversight 5:04 The human knowledge gap 5:37 Concentration of decision-making 6:10 What we still do not know 6:36 What technical researchers can do 6:51 Experiments on what drives AI progress 7:46 Developing metrics for oversight 8:16 More AI R&D evaluations, including safety R&D More AI safety research: https://far.ai Alignment Workshop playlist: https://youtube.com/playlist?list=PLBY5kyt_LfFg&si=0IfDd-WNQwrs14Kn FAR.AI is a research nonprofit working to ensure the safe development of advanced AI. We host the Alignment Workshop series and publish frontier alignment research.