MoRI + vLLM: Wide Expert Parallelism and RDMA KV-Cache Transfer for Disaggregated MoE Serving on AMD

Running large Mixture-of-Experts models across pods creates two major challenges: expert dispatch and combine traffic, and moving the KV cache between prefill and decode. At #PyTorchCon North America 2026, Ravi Gupta, Rishi Madduri, and Shiksha Patel of AMD will present “MoRI + vLLM: Wide Expert Parallelism and RDMA KV-Cache Transfer for Disaggregated MoE Serving on AMD.” The poster will show how bringing the MoRI communication stack into vLLM enables Wide Expert Parallelism across pods using RDMA and moves the KV cache for disaggregated prefill and decode on AMD Instinct GPUs. They will also cover correctness and stability issues encountered when spanning pods and share vLLM benchmark results for long, high-concurrency workloads. Register for PyTorch Conference North America 2026: https://hubs.la/Q04v4SL60