REPA Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You ...
REPA (Representation Alignment) is a clever trick to align diffusion transformers’ representations with pretrained transformers like DINOv2. It accelerates training and improves the diffusion model’s ability to do things other than image generation (such as image classification). Tune in to find out how it works. AI Coffee Break Merch! 🛍️ https://ai-coffee-break-with-letitia.myspreadshop.de/ 📃Yu, S., Kwak, S., Jang, H., Jeong, J., Huang, J., Shin, J. and Xie, S., 2024. “Representation alignment for generation: Training diffusion transformers is easier than you think.” Submitted at ICLR 2025 https://openreview.net/forum?id=DJSZGGZYVi Thanks to our Patrons who support us in Tier 2, 3, 4: 🙏 Dres. Trost GbR, Siltax, Vignesh Valliappan, Michael, Sunny Dhiana, Andy Ma Outline: 00:00 Faster diffusion model training 00:45 Diffusion models and general-purpose visual representations 01:44 DINOv2 explained 02:13 The idea behind REPA 02:54 REPA explained 03:58 Results that speak volumes 05:38 Reflections ▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀ 🔥 Optionally, pay us a coffee to help with our Coffee Bean production! ☕ Patreon: https://www.patreon.com/AICoffeeBreak Ko-fi: https://ko-fi.com/aicoffeebreak Join this channel as a Bean Member to get access to perks: https://www.youtube.com/channel/UCobqgqE4i5Kf7wrxRxhToQA/join ▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀ 🔗 Links: AICoffeeBreakQuiz: https://www.youtube.com/c/AICoffeeBreak/community Twitter / X: https://twitter.com/AICoffeeBreak LinkedIn: https://www.linkedin.com/in/letitia-parcalabescu/ Threads: https://www.threads.net/@ai.coffee.break Bluesky: https://bsky.app/profile/aicoffeebreak.bsky.social Reddit: https://www.reddit.com/r/AICoffeeBreak/ YouTube: https://www.youtube.com/AICoffeeBreak Substack: https://aicoffeebreakwl.substack.com/ #AICoffeeBreak #MsCoffeeBean #MachineLearning #AI #research Video editing: Nils Trost Music 🎵 : Bella Bella Beat – Nana Kwabena