CAP Theorem Reloaded: How to Optimize Distribution of Telemetry Data | RTA Summit 2025

At modern cloud-fleet scale, and as systems grow more complex, the volume of telemetry data in the form of logs, metrics, and traces sent to observability (o11y) systems can quickly exceed manageable limits. This talk dives into tried and true methods for defining and enforcing telemetry data quotas for OpenTelemetry (OTel) Collectors, based on CAP Theorem. We have been applying the principles of CAP Theorem to provide the framework for achieving better telemetry volume & distribution. Through our research & application, we'll share strategies for efficiently measuring data from multiple OTel agents, ensuring rule-based data distribution & addressing the practical implications of the CAP Theorem within telemetry pipelines for cloud native systems. By reframing CAP principles in the context of telemetry data, we'll consider trade-offs between consistency, availability, and partition tolerance when scaling and managing quotas across distributed systems. Join: https://stree.ai/slack | Sub: https://stree.ai/sub | LEARN MORE ► Learn about Apache Pinot: https://dev.startree.ai CONNECT Subscribe on YouTube: https://youtube.com/c/startreedata?sub_confirmation=1 Community Slack: https://stree.ai/slack Twitter: https://twitter.com/startreedata Linkedin: https://www.linkedin.com/company/startreedata GitHub: https://github.com/startreedata Site: https://startree.ai ABOUT STARTREE StarTree, powered by Apache Pinot: real-time analytics for user-facing apps. Discover the latest technical developments, trends, and use cases for real-time analytics straight from our engineers and open source developers, and your fellow industry colleagues and practitioners. StarTree is uniquely positioned to bring you fast, reliable and fresh perspectives on the shift from batch to real-time analytics, and from internal-facing to user-facing applications. Subscribe to our channel if you are designing systems that need to scale to petabytes of information, millions of users, and to provide results at sub-second scale latencies. #apachepinot #realtimeanalytics #startree