AI Showdown: Benchmarking Models for IT and Security Investigations

Whether it is a security alert or a performance degradation, your team is tasked with interpreting fragmented telemetry to uncover the truth under extreme pressure. Investigations are time-consuming and depend on specialized expertise, which is why modern AI solutions for security and IT operations focus heavily on this core capability. From AI SOC to autonomous SRE, the fundamental value lies in an agent's ability to conduct an effective investigation. But with new AI models launching constantly, security and IT teams face a practical challenge: how do you know which models are actually best for your investigations? Traditional AI benchmarks can tell you how well a model codes or handles general reasoning, but not how it will perform when investigating a security incident, troubleshooting an outage, or working with messy, incomplete telemetry. And as teams gain more flexibility to bring their own models and token economics become increasingly important, choosing the right model for the right job matters more than ever. In this session, we’ll introduce SecIT Bench, Cribl’s benchmark for evaluating AI across real-world security and IT scenarios, and show how its findings can help you make smarter AI decisions. You’ll learn how to: - Compare models across accuracy, cost, speed, and efficiency - See where different models excel or struggle across security and observability workflows - Understand how the tooling around a model impacts its performance - Make better model choices based on your workflows, priorities, and budget You’ll leave with a practical framework for looking beyond headline benchmark scores and making more informed decisions about the AI powering your security and IT workflows. ## Follow Cribl LinkedIn: https://www.linkedin.com/company/cribl/ Twitter: https://www.twitter.com/cribl_io Sign up for a Cribl.Cloud account: https://cribl.cloud/signup/ Learn more about Cribl: https://cribl.io