Observability and Evals Masterclass

Every prompt change, model swap, or tool update can introduce new failures. An eval loop helps you check whether your changes improve your agent and catch regressions before they ship. In this workshop, you’ll build the complete eval loop in Mastra, from investigating production failures to testing and verifying fixes. You’ll learn how to: Understand each run: Use traces to investigate quality, latency, and cost down to the model call or tool call responsible. Build datasets from real failures: Turn production mistakes and edge cases into test cases. Compare changes: Use datasets, experiments, and scorers to evaluate prompts, models, and tools. Catch regressions: Add evals to CI/CD to test new changes against failures you’ve already fixed. Let a coding agent test its own fixes: Give it CLI access to traces so it can investigate a failure, change the code, rerun evals, and check the result. Expect real code, a live end-to-end demo, and plenty of time for Q&A.