The Shift from Training to Monetizing Models | Deep Geeks ep03

Memory and flash suppliers are trading like meme stocks. That is not hype, it is a signal about where AI infrastructure is going. https://www.weka.io/podcast/?utm_source=youtube&utm_medium=social The AI industry spent billions training models. It is now shifting to monetizing them, and training and inference are fundamentally different workload profiles. Different enough that you could reasonably build different data centers for each. In this clip from DeepGeeks Episode 3, we get into why inference is a memory centric problem. The rough math: a 100,000 token prompt, roughly a megabyte of data, used to translate into 30 to 50 gigabytes of expensive high bandwidth memory. Recent innovations have compressed that to 3 to 5 gigabytes. But Jevons paradox does what it always does. Shrink the unit and the number of units explodes, and the industry ends up memory bound anyway. The tough part is the supply chain. By some analyst accounts there are not enough rare earth minerals or fabrication capacity in the world to produce the memory that demand now calls for. That is the memory wall, and getting past it takes a cleverer approach than buying more. *Learn more:* *Listen to the full episode:* https://www.weka.io/podcast/?utm_source=youtube&utm_medium=social&utm_content=full-episode *WEKA NeuralMesh:* https://www.weka.io/product/neuralmesh/?utm_source=youtube&utm_medium=social&utm_content=product-page *Subscribe for more AI infrastructure content:* https://www.youtube.com/@WekaIO?sub_confirmation=1