Catherine Olsson and Nelson Elhage: Anthropic, Understanding Transformers
The Gradient: Perspectives on AI
How to Replicate the Induction Heads Findings
There's some math, but you could skip it. There's a lot of sort of empirical results that is very easy to kind of take in. And I do think you'll get a better understanding of first-homeers if you take a read. We've had groups at other research labs, both in industry and academia, replicate our findings. Another type of investigation that you can do as well are just qualitatively look at what neurons and attention heads are doing.
00:00
Transcript
Play full episode
Remember Everything You Learn from Podcasts
Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.
