The Inside View

Neel Nanda on mechanistic interpretability, superposition and grokking

4 snips
Sep 21, 2023
Guest
Neel Nanda
Neel Nanda, a researcher at Google DeepMind, discusses mechanistic interpretability in AI, induction heads in models, and his journey into alignment. He explores scalable oversight, the ambitious degree of interpretability in transformer architectures, and the capability of humans to understand complex models. The podcast also covers linear representations in neural networks, the concept of superposition in models and features, Terry Matt's mentorship program, and the importance of interpretability in AI systems.
Ask episode
Chapters
Transcript
Episode notes
1
Introduction
00:00 • 3min
2
Mechanistic Interpretability in AI
03:29 • 23min
3
Scalable Oversight and Giving Feedback to Language Models
26:01 • 3min
4
Exploring the Ambitious Degree of Interpretability in Current Transformer Architectures
28:31 • 2min
5
The Capability of Humans to Understand Complex Models and Algorithms
30:12 • 24min
6
Linear Representations and Feature Extraction in Neural Networks
54:21 • 21min
7
Superposition in Models and Features
01:14:52 • 11min
8
Terry Matt's: Aligning Talented Individuals
01:25:40 • 6min
9
Generalization of Tiny Models on Algorithmic Tasks
01:32:05 • 22min
10
Discussion on YouTube content and research walkthroughs
01:54:18 • 2min
11
The Importance of Interpretability in AI Systems
01:55:58 • 5min
12
Exploring Mechanistic Interpretability and Its Applications
02:00:38 • 4min