Future of Life Institute Podcast cover image

Neel Nanda on Avoiding an AI Catastrophe with Mechanistic Interpretability

Future of Life Institute Podcast

00:00

Reverse Engineering Induction Heads

induction heads are a general algorithm that can work on arbitrary words, but which does this very specific task. They appear in all the open source models I could find and they're really important for what we call in context learning. The ability of models to do this perfectly coincides with the dramatic bit where they're learnt.

Transcript
Play full episode

The AI-powered Podcast Player

Save insights by tapping your headphones, chat with episodes, discover the best highlights - and more!
App store bannerPlay store banner
Get the app