How Do You Evaluate an Open Frontier Model? — David Hall & Braden Hancock

Frontier Data Summit 2026: Braden Hancock hosts David Hall of Marin to explain how they evaluate an open frontier language model as it is trained in public. Marin is building a frontier open language model in public. In this conversation, the team explains how they evaluate models during pre-training and post-training, using loss, scaling laws, synthetic evaluations, and benchmarks such as Terminal-Bench. They also discuss data contamination, alignment, and the need for more public post-training data and environments. IN THIS VIDEO • Why Marin is building a frontier model in the open • Evaluating a model early in pre-training with scaling laws and loss • When traditional benchmarks become useful • Synthetic evaluations and long-context ability • Data contamination, alignment, and open post-training SPEAKERS David Hall, Member of Technical Staff, Open Athena (Marin) Braden Hancock (host), AI Researcher & Investor; Partner, Laude Institute CHAPTERS 00:00 Marin’s open-model mission 01:17 Why the project began 02:48 The large-scale training run 04:02 Early evaluations, scaling laws, and loss 06:18 When traditional benchmarks become useful 08:35 Synthetic evaluations and long-context ability 10:18 Benchmark scores, diagnostics, and benchmark size 12:12 Favorite, overrated, and missing benchmarks 15:26 Audience Q&A: building Marin in the open 17:47 Scaling laws and data contamination 20:00 Alignment, data quality, and open post-training RESOURCES Marin project: https://marin.community/ Marin code and experiments: https://github.com/marin-community/marin Snorkel AI Open Benchmarks Grants: https://benchmarks.snorkel.ai/