Future of Life Institute Podcast cover image

Neel Nanda on Avoiding an AI Catastrophe with Mechanistic Interpretability

Future of Life Institute Podcast

CHAPTER

How to Predict What Comes After Apple?

The model is trying to predict what comes next and say the next word in its training data is tree. It turns out there are some heads which learn to look at the word tower. And yeah, you often get heads for dramatic structures like look at the first word in the sentence, look at the subject of the current sentence, etc. The same heads will do things do this for the looks things like the Colosseum is located in or the Parthonon is located in.

00:00
Transcript
Play full episode

Remember Everything You Learn from Podcasts

Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.
App store bannerPlay store banner