How to Build a Model Router in the Harness

Sydney Runkle, product manager for LangChain's open source team, walks through how the team built a model router into Open SWE, LangChain's open source coding agent. She covers the four steps: mapping the task mix from LangSmith traces, picking models along the Pareto frontier, building the router in the harness, and tracking outcomes. In an A/B test against always using the performance model, the router cut median cost per thread by 64% with no measurable change in quality. Chapters: 0:00 Frontier intelligence is expensive at scale 1:24 What a model router actually does 1:56 The result: 64% lower median cost per thread 2:10 Four steps to build a model router 2:40 Step 1: Map your agent's tasks with LangSmith traces 4:53 Step 2: Pick models along the Pareto frontier 5:59 Step 3: Build the router in the harness 6:49 Swapping the classifier to Jev 7:23 Why open models fit the fast tier 7:46 Step 4: Track outcomes with evals or an A/B test 8:49 Results: 64% lower median cost, no drop in merged PRs 10:14 Why we stopped the fast tier test after a day 10:45 What's next: benchmarks, subagents, mid thread rerouting 12:30 How to add a router to your own agent Resources: → Companion blog, How to Build a Model Router in the Harness: https://www.langchain.com/blog/how-to-build-a-model-router-in-the-harness → Building a Harness with Jev: https://www.langchain.com/blog/building-a-harness-with-jev → Open SWE: https://github.com/langchain-ai/open-swe → LangSmith: https://www.langchain.com/langsmith