Future of Life Institute Podcast cover image

Neel Nanda on Avoiding an AI Catastrophe with Mechanistic Interpretability

Future of Life Institute Podcast

00:00

How to Predict What Comes After Apple?

The model is trying to predict what comes next and say the next word in its training data is tree. It turns out there are some heads which learn to look at the word tower. And yeah, you often get heads for dramatic structures like look at the first word in the sentence, look at the subject of the current sentence, etc. The same heads will do things do this for the looks things like the Colosseum is located in or the Parthonon is located in.

Transcript
Play full episode

The AI-powered Podcast Player

Save insights by tapping your headphones, chat with episodes, discover the best highlights - and more!
App store bannerPlay store banner
Get the app