High Performance Serving Framework for Offline and Real-time Diffusion Models - SGLang-Diffusion

SGLang-Diffusion is a high-performance serving framework for diffusion models, designed for both large-scale offline generation and latency-sensitive real-time inference. As diffusion workloads expand from image and video generation to diffusion-based world models, serving systems must handle very different constraints: maximizing GPU throughput for batch/offline jobs, while maintaining tight end-to-end latency for interactive and closed-loop environments. At PyTorch Conference North America 2026, Yihao Wang and Kevin Mi of RadixArk will present the design of SGLang-Diffusion and the PyTorch-based inference optimizations behind it, including efficient request scheduling, memory management, batching strategies, and execution-path optimizations for diffusion pipelines. They will discuss how serving diffusion models differs from serving LLMs, why existing inference systems are often insufficient for iterative denoising workloads, and how a unified framework can support both offline generation and real-time world-model inference. Join us in San Jose, October 20-21: https://hubs.la/Q04v4SL60 #PyTorchCon