Training large language models to reason in a continuous latent space – COCONUT Paper explained
AI doesn’t have to think with words. We explain COCONUT (Chain of Continuous Thought) 🥥, a new paper that makes Chain-of-Thought work with vectors instead of words like traditional CoT does. We break down: ☕ Why traditional Chain of Thought (CoT) reasoning might not be optimal. ☕ How COCONUT uses continuous vectors for CoT instead of human language and makes CoT faster. ☕ What this means for AI interpretability and scalability. AI Coffee Break Merch! 🛍️ https://ai-coffee-break-with-letitia.myspreadshop.de/ Thanks to our Patrons who support us in Tier 2, 3, 4: 🙏 Dres. Trost GbR, Siltax, Vignesh Valliappan, Michael, Sunny Dhiana, Andy Ma 📃 Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason Weston, and Yuandong Tian. "Training large language models to reason in a continuous latent space." (2024). https://arxiv.org/abs/2412.06769 Outline: 00:00 Continous CoT explained 00:46 Limits of Chain-of-Thought 01:34 CoT in Latent space 02:27 Normal LLM CoT 02:52 Coconut explained 🥥 05:31 Results 07:14 Understanding what’s inside of continuous CoT ▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀ 🔥 Optionally, pay us a coffee to help with our Coffee Bean production! ☕ Patreon: https://www.patreon.com/AICoffeeBreak Ko-fi: https://ko-fi.com/aicoffeebreak Join this channel as a Bean Member to get access to perks: https://www.youtube.com/channel/UCobqgqE4i5Kf7wrxRxhToQA/join ▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀ 🔗 Links: AICoffeeBreakQuiz: https://www.youtube.com/c/AICoffeeBreak/community Twitter / X: https://twitter.com/AICoffeeBreak LinkedIn: https://www.linkedin.com/in/letitia-parcalabescu/ Threads: https://www.threads.net/@ai.coffee.break Bluesky: https://bsky.app/profile/aicoffeebreak.bsky.social Reddit: https://www.reddit.com/r/AICoffeeBreak/ YouTube: https://www.youtube.com/AICoffeeBreak Substack: https://aicoffeebreakwl.substack.com/ Web: https://explanationmark.de/letitia https://aicoffeebreak.com #AICoffeeBreak #MsCoffeeBean #MachineLearning #AI #research Video editing: Nils Trost