The Role Of Neural Geometry In Interpretability | Atticus Geiger (Goodfire)

Atticus Geiger (Goodfire) on how neural networks represent concepts, and why the concepts-as-directions view behind sparse autoencoders is incomplete. Geiger argues that the linear picture behind sparse autoencoders, where each concept is a single direction, misses the multidimensional geometry inside neural networks. He frames interpretability as tracing real-world conceptual structure into training-data statistics and then into a model's representations, algorithms, and behavior. His examples: months like "April" form circles inside a model, the geometry of human emotions reappears in activations, and "manifold steering" controls behavior more precisely than linear steering. In one case, a model builds an internal calculator from Fourier features (in Llama 3.1 8B) to reason about time. He then examines how sparse autoencoders can shatter, compactly capture, or only partly represent this geometry, and sketches a path to automate interpretability, moving from dictionaries to encyclopedias with AI research agents. Chapters 0:00 How do neural networks represent concepts? 1:02 Beyond the linear, sparse-autoencoder view 2:07 The big-picture view of interpretability 2:50 How April gets inside a neural network 3:55 Emotional dynamics: behavior meets representation 5:07 Manifold steering vs. linear steering 7:03 An internal calculator and Fourier features 8:35 How sparse autoencoders capture geometry 10:28 Discovering new manifolds, unsupervised 10:42 From dictionaries to encyclopedias 12:22 Automating interpretability with research agents More AI safety research: https://far.ai Alignment Workshop playlist: https://youtube.com/playlist?list=PLBY5kyt_LfFg&si=xAea0v-rwGScP3zB FAR.AI is a research nonprofit working to ensure the safe development of advanced AI. We host the Alignment Workshop series and publish frontier alignment research.