PyTorch Day India 2026 Scaling Mixture of Experts for Everyone: High Performance MoE Training in PyT

Mixture-of-Experts (MoE) architectures offer a powerful path to scaling model capacity without proportional increases in compute — but training them efficiently at large scale has traditionally required complex distributed systems expertise. This talk walks through how developers can scale MoE models directly in PyTorch, using distributed features and optimized parallelism strategies