Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution – Paper Explained
We've combed through the complex mathematics and dense pages of the “Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution” research paper to bring you the essential insights and key takeaways. Learn about how diffusion models can finally generate good quality text. The paper won the #ICML2024 best paper award! 👏 ✨ Advance your career in Artificial Intelligence with Simplilearn’s Artificial Intelligence Engineer Program: https://bit.ly/AICoffeeBreak-Simplilearn (sponsor) AI Coffee Break Merch! 🛍️ https://ai-coffee-break-with-letitia.myspreadshop.de/ This video in blog post format! 👉 https://aicoffeebreakwl.substack.com/p/discrete-diffusion-modeling-by-estimating Thanks to our Patrons who support us in Tier 2, 3, 4: 🙏 Dres. Trost GbR, Siltax, Vignesh Valliappan, Michael, Sunny Dhiana, Andy Ma 📺 Our diffusion models explained: https://www.youtube.com/playlist?list=PLpZBeKTZRGPPvAyM9DM-a6W0lugCo8WfC Outline: 00:00 Intro 01:17 Simplilearn (Sponsor) 02:29 Impossible for GPT, but possible with diffusion 04:26 Discrete Diffusion 05:33 Similarity to BERT 06:50 Forward diffusion 07:48 Backward diffusion 08:44 Score Entropy for learning 09:17 Inference: Generating a sample 09:52 Results 📃Lou, Aaron, Chenlin Meng, and Stefano Ermon. "Discrete diffusion language modeling by estimating the ratios of the data distribution." (2023) https://arxiv.org/abs/2310.16834 📺 Talk by Aaron Lou about Discrete Diffusion Modelling: https://youtu.be/_1qv_LNjH9U ▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀ 🔥 Optionally, pay us a coffee to help with our Coffee Bean production! ☕ Patreon: https://www.patreon.com/AICoffeeBreak Ko-fi: https://ko-fi.com/aicoffeebreak Join this channel to get access to perks: https://www.youtube.com/channel/UCobqgqE4i5Kf7wrxRxhToQA/join ▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀ 🔗 Links: AICoffeeBreakQuiz: https://www.youtube.com/c/AICoffeeBreak/community Twitter: https://twitter.com/AICoffeeBreak Reddit: https://www.reddit.com/r/AICoffeeBreak/ YouTube: https://www.youtube.com/AICoffeeBreak #AICoffeeBreak #MsCoffeeBean #MachineLearning #AI #research Video editing: Nils Trost Music 🎵 : Space Navigator - Sarah, The Illstrumentalist