The Science of Reverse Engineering Neural Networks (Intro to Mechanistic Interpretability)

A practical guide to neural nets, superposition, and expts around polysemanticity. Try Explain by Scrimba: https://scrimba.com/explain?via=avb Preparing a Github repo for all my experiments, will share soon. Follow me on X for updates about upcoming videos: https://x.com/neural_avb To support the channel, join our Patreon: https://www.patreon.com/NeuralBreakdownwithAVB Members get access to everything behind-the-scenes that goes into producing my videos - including code. Plus, it supports the channel in a big way and helps to pay my bills. References: Neel Nanda's reading list: https://www.alignmentforum.org/posts/NfFST5Mio7BCAQHPA/an-extremely-opinionated-annotated-list-of-my-favourite-1 Transformer Circuits Thread: https://transformer-circuits.pub/ Individual papers related to this video: 1. Toy Models of Superposition https://arxiv.org/abs/2209.10652 2. Zoom In: An Introduction to Circuits. https://distill.pub/2020/circuits/zoom-in/ 3. The Linear Representation Hypothesis https://arxiv.org/abs/2311.03658 4. What Causes Polysemanticity? https://arxiv.org/abs/2312.03096 5. Superposition, Memorization, and Double Descent. https://transformer-circuits.pub/2023/toy-double-descent/index.html 6. Tracing the thoughts of a language model: https://www.anthropic.com/research/tracing-thoughts-language-model 0:00 - Intro to Mechanstic Interpretability 07:21 - MLP layers, Monosemantic vs Polysemantic Neurons 15:07 - Antipodal pairs, Linear vs ReLU 21:10 - Superposition 30:10 - How Bias helps 35:52 - Sparsity vs Feature Importance 43:17 - ReLU at the hidden layer, Privileged Basis 54:32 - How might neurons store facts 1:03:30 - Forcing sparse neurons