RLlib: Lessons from the V2 Stack and Road Ahead | Ray Summit 2025

Slides: https://drive.google.com/file/d/1x1kn2_X46xr-BYYcnVtM3KDT8PZsl2tw/view?usp=sharing At Ray Summit 2025, Artur Niederfahrenhorst and Simon Lars Zehnder from Anyscale share the major milestones behind the release of the RLlib v2 stack, now GA and redesigned for the next generation of large-scale reinforcement learning workloads. They walk through the key learnings that shaped the new RLlib v2 architecture, detailing the engineering improvements that boost scalability, reliability, and extensibility. The session highlights major enhancements enabling RLlib to scale to 10,000+ environment runners and 100+ learners, supporting massive distributed RL training with high throughput. Artur and Simon also outline the road ahead for RLlib, including deeper integrations with a wide range of simulators and expanded capabilities for complex, simulation-heavy RL applications. Whether you're building large-scale RL systems, accelerating research pipelines, or deploying RL in production, this session offers a clear view into the future of RLlib and how it’s evolving to meet modern AI demands. Liked this video? Check out other Ray Summit breakout session recordings https://www.youtube.com/playlist?list=PLzTswPQNepXllnU0C36WtkC0dqkAoDulh Subscribe to our YouTube channel to stay up-to-date on the future of AI! https://www.youtube.com/c/anyscale 🔗 Connect with us: LinkedIn: https://www.linkedin.com/company/joinanyscale/ X: https://x.com/anyscalecompute Website: https://www.anyscale.com/