Build A Reasoning Model From Scratch 4: Inference Scaling 1 (Temperature, Top-p, Self-Consistency)

After motivating the use cases for inference-time scaling, this video first walks through temperature scaling, top-p filtering, and multinomial sampling to generate diverse answers with an LLM. Then, using these techniques, we implement self-consistency to improve LLM answer accuracy. - Reasoning playlist: https://www.youtube.com/watch?v=Kh9mqTzjuEQ&list=PLfrHAQCRxtUs - Reasoning Book: https://amzn.to/4aAKiFY - Reasoning GitHub repo: https://github.com/rasbt/reasoning-from-scratch - LLMs from Scratch book: https://amzn.to/4fqvn0D - LLMs from Scratch repo: https://github.com/rasbt/LLMs-from-scratch - LLMs from Scratch playlist: https://www.youtube.com/watch?v=yAcWnfsZhzo&list=PLTKMiZHVd_2IIEsoJrWACkIxLRdfMlw11&index=1 00:00 Introduction and recap 00:31 Training-time and inference-time scaling 07:52 What we'll implement 11:47 Notebook setup and model loading 17:43 Building a flexible text generation function 24:40 Chain-of-thought prompting 28:26 Sampling and output diversity 33:43 Next-token logits and greedy decoding 38:20 Temperature scaling step by step 42:46 Softmax and token probabilities 47:42 Multinomial sampling 54:51 Adding temperature sampling to text generation 59:31 Top-p filtering step by step 1:10:23 Adding top-p filtering to text generation 1:13:43 Sampling and LLM watermarking 1:16:01 Self-consistency and majority voting 1:20:36 Implementing self-consistency 1:29:02 MATH-500 results 1:35:01 Accuracy and compute tradeoffs 1:36:50 Next steps and self-refinement