No more black boxes: designing for trust in enterprise AI

Every enterprise has an AI mandate. But how do you build trust and transparency into AI tools and platforms? Celigo's Head of User Experience, Susanne, has spent the last several months on that problem. When Celigo tested an early Agent Builder prototype that took action without showing its reasoning, it didn't feel powerful to anyone. Developers and business users alike came away anxious. That finding reshaped how every agent interface at Celigo gets built. In this episode, Susanne names the principles behind that work and explains where each one came from. The six principles she covers: ▸ Everyone gets supported. The same platform has to work for a developer who wants full control, a mid-tier builder who wants a framework, and a non-technical user watching an error queue. None of them should hit a screen they can't read. ▸ Trust is built, not granted. "Trust is not automatically given up front. It's built through the small moments of interaction and clarity." Moving from configuration screens to natural language doesn't change that. ▸ The user stays in the driver's seat. Customers should be able to adopt new automation paradigms without giving up control of what runs in their business. ▸ Transparency is a foundation, not a nice-to-have. In validation sessions, not one customer — technical or not — asked for an agent that acts without showing why. Agent Trace exists because of that: a running journal of the inputs an agent read, the tools it called, the decisions it made, the raw model response, and the final output. ▸ Consistency is a trust mechanism. The import/export mental model gets reused across the platform on purpose. Learn app integration and you already know how to do EDI — same patterns, same buttons, same expectations on every screen. ▸ Strong design makes complexity invisible. A thousand connectors, deterministic workflows, and AI agents in one platform, and the person using it shouldn't have to feel any of that. CHAPTERS 00:00 Introducing Susanne 01:30 Principle one: every user type should feel supported, from developer to error-watcher 04:00 Principle two: trust is built through small moments of clarity, not granted upfront 04:45 Principle three: the user stays in the driver's seat as the paradigm shifts 05:30 Agent Trace: the inputs, tools, decisions, raw model response, and output an agent produces 08:30 Principle four: nobody asked for a black box, in any validation session 10:00 How Celigo runs user research before a screen gets designed 12:00 Designing one product for a developer and a non-technical user at the same time 14:00 Principle five: reusing the import/export mental model as a trust mechanism 16:00 First principles versus a design system when paradigms change every quarter 18:30 When a PMM shows up with an AI-built prototype instead of a requirements doc 21:00 Design skills that catch accessibility gaps and inconsistent labels in a live review 23:30 The tension: validate what customers asked for, or bet on what they can't imagine yet 26:00 Chat interface or form fields — the answer depends which customer you ask 28:30 Why faster tools mean more space for design judgment, not less 30:00 Principle six: strong design makes complexity invisible 32:30 What Susanne is building right now 🎙️ Listen to Builder's Hub Podcast: https://celigo.com/podcast 👉 Get a demo: https://www.celigo.com/request-a-demo/