Google Just Cracked NVIDIA's Biggest Moat For its TPUs

For years, NVIDIA's biggest competitive moat wasn't just silicon—it was the code written on top of it. If your PyTorch models were covered in CUDA references, running on Google's TPUs meant rewriting your entire architecture in JAX or fighting PyTorch/XLA's painful lazy execution graphs and manual mark_step boundaries. At PyTorch Conference 2026, Google and Meta's PyTorch team open-sourced TorchTPU. The pitch sounds almost too simple: replace model.to("cuda") with model.to("tpu"), and your existing code runs natively on Google's chips. In this deep dive, we open the arXiv paper, technical documentation, and community benchmarks to see if that promise holds up. We break down the "DeferAndFuse" engine that converts eager PyTorch operations into compiled StableHLO programs, why 98% of popular Hugging Face models passed testing, and the hidden performance taxes Google left in the fine print. ⏱️ Chapters: 0:00 The One-Line Promise: .to("tpu") 1:26 The Death of PyTorch/XLA 2:37 Why Meta Built a Backend for Google TPUs 3:44 Inside the Engine: Dispatcher & DeferAndFuse 8:02 Does It Actually Work? (98% Pass Rate Tested) 10:24 The Real Benchmark: 52% vs 27% MFU 12:44 What Actually Cracked in NVIDIA's Moat 🔗 Sources & Documentation: • Google Cloud Blog - TorchTPU Announcement: https://cloud.google.com/blog/topics/ai-infrastructure/torchtpu-is-now-open-source-run-native-pytorch-on-tpus • TorchTPU Technical Paper (arXiv 2610.24161): https://arxiv.org/html/2610.24161v2 • Google Developer Blog - TorchTPU at Scale: https://developers.googleblog.com/en/torchtpu-running-pytorch-natively-on-tpus-at-google-scale/ • Google Cloud TPU PyTorch Documentation: https://docs.cloud.google.com/tpu/docs/pytorch-on-tpu • Official TorchTPU GitHub Repository: https://github.com/google-pytorch/torch_tpu • PyTorch/XLA Deprecation & Archival Notice: https://github.com/pytorch/xla • Hugging Face Optimum-TPU Archive: https://huggingface.co/docs/optimum-tpu/index • vLLM TorchTPU Backend Plugin: https://github.com/vllm-project/vllm-torchtpu • Tenstorrent Libtt PJRT Plugin: https://github.com/pcmoritz/libtt • Hacker News Discussion: https://news.ycombinator.com/item?id=50169053 🔔 Subscribe: https://www.youtube.com/channel/UC0DZj1PNa_Fp0MU6uPSKv5w?sub_confirmation=1 💙 Become a Member: https://www.youtube.com/channel/UC0DZj1PNa_Fp0MU6uPSKv5w/join 🐦 Twitter/X: https://x.com/cloud_codes 💬 Discord: https://discord.gg/mhgXXmhrh #pytorch #nvidia #google #tpu #cuda #machinelearning #deeplearning #ai #cloudcodes