Fireside Chat: Increase Accuracy and Scale Evaluations with Centor

In this fireside chat, data scientists Joshua Rubin and Jennifer Flynn explore why evaluating AI agents is harder than testing conventional software. Agent inputs and outputs are unstructured and non-deterministic, teams need dedicated evaluators, and specialist models to handle safety, faithfulness, and PII detection faster and more affordably than generalist LLMs. Fiddler Centor runs these specialist models directly within your deployment. Dynamic Model Escalation routes edge cases to a larger generalist LLM, improving accuracy while keeping costs low, and data secure within your boundary. The conversation also covers tuning evaluation thresholds, building custom judges from annotated golden datasets, and applying universal evaluators across every agent, giving security leaders and AI executives a single control plane for their growing agent portfolio. Learn how Fiddler Centor scales evaluations for your agentic AI deployments: https://www.fiddler.ai/blog/dynamic-model-escalation-ai-evals