
19 - Mechanistic Interpretability with Neel Nanda
AXRP - the AI X-risk Research Podcast
Induction Heads Are Relevant to Context Learning?
induction heads are actually a deep and fundamental mechanism of how models do things. In context learning you're going to distill out the bits in each like section of the text into what exactly is going on here. There's kind of two ways you could implement an algorithm like this one of them is asymmetric when you learn a lookup table. The other kind of thing you can do is symmetric where you say if a is relevant to b then b is also relevant to a.
00:00
Transcript
Play full episode
Remember Everything You Learn from Podcasts
Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.