Catherine Olsson and Nelson Elhage: Anthropic, Understanding Transformers
The Gradient: Perspectives on AI
Induction Heads
Induction heads looks at this pot token, POT and then finds a previous instance of potters. It's kind of like copy pasting from the past to make things easy. And I think importantly, they're doing it in the present context.
00:00
Transcript
Play full episode
Remember Everything You Learn from Podcasts
Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.
