INSANE: This 46 MB AI Model Runs 5x FASTER on My Laptop (Frozen Embeddings TESTED)

A new paper (arXiv 2610.22428) says the learned embedding table in small language models is overpriced: replace it with a frozen 16 MB table of 512-bit codes and the model matches or beats the learned one on three of four benchmarks, a ternary version fits in 46 MB, and new words can be added after training at a 3% cost instead of 43%. I read the paper, pulled the repo, built the authors' CPU kernel on my own Windows laptop (no C compiler installed, so I used Zig), and benchmarked it against the learned-table model in llama.cpp. My measured results (CPU only, 8 threads, 128 prefill + 64 generate): - Frozen ternary kernel: 517 tok/s, 63 MB peak RAM, 46.2 MB weights - Learned table, Q8_0 GGUF in llama.cpp: 103 tok/s (about 5x slower) - Kernel next-token loss on the authors' 64 validation tokens: 4.4947 (paper: 4.495) - Paper's numbers on its Xeon: 615-740 tok/s in 62 MB Not tested by me: benchmark accuracies, training, and the new-word results. Those are the paper's numbers. Chapters 0:00 The dictionary problem 0:33 Where the parameters live 1:00 The frozen 512-bit table 1:46 The three-arm experiment 2:18 What the learned table buys 3:10 I test it on my laptop 4:27 Adding new words after training 5:00 Limits and license 5:29 Verdict Links Model + code + data: https://huggingface.co/QLNI/embeddings-are-overpriced Paper: https://arxiv.org/abs/2610.22428 llama.cpp: https://github.com/ggml-org/llama.cpp RunPod: https://get.runpod.io/pe48 License note: non-commercial research only; the table and checkpoints of arms B and C also fall under Google's Gemma terms. AI-generated imagery is used for atmosphere and narration is a cloned voice of the channel owner. Get GPUs Runpod: https://get.runpod.io/pe48 Get Hostinger: https://www.hostg.xyz/SHJiu Coupon Code "PROMPT" 10% discount 🔗 Connect with me: Email → [email protected] Ko-fi → https://ko-fi.com/promptengineer Patreon → https://www.patreon.com/PromptEngineer975 Book a Call → https://calendly.com/prompt-engineer48/call GitHub → https://github.com/PromptEngineer48 Twitter/X → https://twitter.com/prompt48 Website → https://promptengineer48.com/