Catherine Olsson and Nelson Elhage: Anthropic, Understanding Transformers
The Gradient: Perspectives on AI
00:00
How to Replicate the Induction Heads Findings
There's some math, but you could skip it. There's a lot of sort of empirical results that is very easy to kind of take in. And I do think you'll get a better understanding of first-homeers if you take a read. We've had groups at other research labs, both in industry and academia, replicate our findings. Another type of investigation that you can do as well are just qualitatively look at what neurons and attention heads are doing.
Transcript
Play full episode
