DeepSeekV4 - Manifold Constrained Hyper Connections (mHC) and the evolution of ResNets

DeepSeek's new mHC: Manifold-Constrained Hyper-Connections is one of the top trending papers in the ML space currently - and this video visually breaks down everything you need to know about it. We talk about Residual connections, their role in Deep Learning, the idea of Hyper-Connections, and build intuition about the new Manifold-Constraint idea that is introduced in this paper. I have tried to use as many visualizations and illustrations to break down these concepts - feel free to pause, rewind, and rewatch sections that require extra attention! This is a very simple optimization on the surface, but the concepts are still pretty deep! You can read the papers here: MHC: https://paperbreakdown.com/abs/2512.24880 HC: https://paperbreakdown.com/abs/2409.19606 Resnets: https://paperbreakdown.com/abs/1512.03385 Check out Paper Breakdown - a website I am solo building where you can discover and study ML research papers. We got a decent free tier to get you started, but here is a 50% discount coupon for the first 10 users who can claim it - MHC50 Follow me on Twitter: https://x.com/neural_avb To join our Patreon, visit: https://www.patreon.com/NeuralBreakdownwithAVB Members get access to everything behind-the-scenes that goes into producing my videos - including code and visualizations from this video. Plus, it supports the channel in a big way and helps to pay my bills. Timestamps: 0:00 - Intro 1:51 - Language Models 4:29 - Residual Connections 5:51 - Identity Mapping 8:47 - Hyper Connections 14:43 - Signal Gain 16:30 - Manifold Constraints and the Birkhoff Polytope #deeplearning #deepseek #neuralnetworks