Live coding Jev from Scratch | Understanding Qwen architecture

Deep dive into Pytorch and HF Transformers, to adapt a Qwen model into JEV. Main YT video: https://youtu.be/sF3CNPbWA8o Git: https://github.com/avbiswas/bev-train We made it choice-order invariant by hacking directly into the attention mask and positional-ids! To join our Patreon, visit: https://www.patreon.com/NeuralBreakdownwithAVB 0:00 Introduction/Advice for new devs 10:05 Jev project 13:21 Decision data set creation 17:16 Understanding model inputs 23:54 Language model architecture 35:29 Data structure and training 51:57 Live coding: Model setup 59:43 Input ids and attention masks 1:13:56 Positional indexing strategy 1:30:31 Handling choice order invariance 2:04:11 Attention mask implementation 2:52:46 Final architecture and testing