
Neel Nanda on Avoiding an AI Catastrophe with Mechanistic Interpretability
Future of Life Institute Podcast
How to Predict What Comes After Apple?
The model is trying to predict what comes next and say the next word in its training data is tree. It turns out there are some heads which learn to look at the word tower. And yeah, you often get heads for dramatic structures like look at the first word in the sentence, look at the subject of the current sentence, etc. The same heads will do things do this for the looks things like the Colosseum is located in or the Parthonon is located in.
00:00
Transcript
Play full episode
Remember Everything You Learn from Podcasts
Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.