What AI Teams Need to Know Before Shipping Agentic Systems

Your agent works. That is not the problem. The problem is that you cannot say why it works, which inputs break it, or what it did the last thousand times a real user touched it. Jacques Verré, Head of Product at Comet, builds Opik, an open-source platform for LLM evaluation and observability. He works with teams who run agents at scale, and the pattern repeats. The demo passes. The team ships. Then a customer reports something strange, and nobody can reconstruct what happened — because an agent failure is rarely a stack trace. It is a tool called with the wrong argument, a retry loop, a confident wrong answer. This talk covers what a team must put in place before it ships: an evaluation suite that reflects real use, tracing that captures every step and not just the final answer, and Opik's Diagnostics, which scans production traces and groups recurring failures into ranked issues with a cause, a suggested fix, and example traces to open. You will leave with a checklist, not a theory. Key takeaways - Working is not the same as understood. You need evidence, not a passing demo. - Build the evaluation suite before you ship, not after the first incident. - Agent failures are rarely exceptions. Most are silent: bad tool arguments, retry loops, confident wrong answers. - Tracing creates a second problem — too much data for a human to read. Let the platform group and rank the failures. - Decide which failure classes need a human in the loop before a user finds them.