Train-to-Test (T²) Scaling Laws | Snorkel AI Reading Group

At our latest Snorkel AI Reading Group, Nicholas Roberts presents Train-to-Test (T²) Scaling Laws — work heading to COLM 2026 that asks what happens to compute-optimal pre-training once you account for the cost of reasoning at inference time. Chinchilla trades model size against training tokens and lands everyone on roughly 20 tokens per parameter, but it says nothing about inference. T² adds a power law in k, holds inference cost constant across model sizes, and concludes you should overtrain dramatically. Mid-paper, Liquid AI released LFM 2.5 — 350M parameters trained on 28 trillion tokens, about 80,000 tokens per parameter — landing almost exactly where T² predicted. The T²-selected checkpoint almost always significantly outperforms a Chinchilla-optimal one at matched inference budget, and the result survives fine-tuning and SFT. The talk covers pre-training scaling laws from first principles, why parallel sampling is the tractable test-time method to analyze, and a Q&A that pushes hard on the modeling assumptions. Chapters 00:00 Introduction 00:28 Roadmap 01:27 What are scaling laws? 03:25 Chinchilla and the 20-tokens-per-parameter rule 05:35 But everyone is doing reasoning now 06:14 Test-time scaling techniques 07:06 Parallel sampling and pass@k 08:15 Train-to-Test (T²) scaling 09:43 Adding a third term: a power law in k 10:24 Inference-cost matching across model sizes 12:03 Why k can go inside a model of the loss 12:45 A second model: predicting accuracy directly 13:10 Two models — loss and pass@k 14:43 The forecast looks radically different 15:00 The catch: we might run out of data 15:34 Liquid AI's LFM 2.5 lands mid-paper 16:06 80,000 tokens per parameter 17:45 How good is the scaling model? 18:38 T² vs. Chinchilla-optimal, head to head 19:13 What happens to the IsoFLOP curves 20:14 The post-training objection 21:34 Results hold after fine-tuning and SFT 22:21 The first end-to-end scaling law 23:01 Q&A 🔗 Links 📄 Paper: https://arxiv.org/abs/2604.01411 📰 VentureBeat coverage: https://venturebeat.com/orchestration/train-to-test-scaling-explained-how-to-optimize-your-end-to-end-ai-compute-budget-for-inference 👤 Nicholas Roberts: https://nick11roberts.science/ 🐦 Nicholas Roberts on X: @nick11roberts 📖 Blog post with full transcript: https://snorkel.ai/blog/train-to-test-scaling-laws-reading-group/ 🎙️ Reading Group info: https://snorkel.ai/reading-group/ Previous Reading Group sessions: ▶️ Senior SWE-Bench: Evaluating Coding Agents — https://snorkel.ai/blog/senior-swe-bench-evaluating-coding-agents-like-senior-engineers/ ▶️ Agents' Last Exam — https://snorkel.ai/blog/agents-last-exam-reading-group/ ▶️ JudgmentBench: Rubric vs. Preference Evaluation — https://snorkel.ai/blog/judgment-bench-comparing-rubric-and-preference-evaluation-for-quality-assessment-legal-ai/ ▶️ Collaborative Gym: Human-Agent Collaboration — https://snorkel.ai/blog/collaborative-gym-a-framework-for-enabling-and-evaluating-human-agent-collaboration/ ▶️ Code World Models & AutoHarness — https://snorkel.ai/blog/code-world-models-and-autoharness-for-llm-agents/ ▶️ Olmix: Data Mixing for LM Development — https://snorkel.ai/blog/olmix-data-mixing-reading-group/