AXRP - the AI X-risk Research Podcast cover image

19 - Mechanistic Interpretability with Neel Nanda

AXRP - the AI X-risk Research Podcast

CHAPTER

How Scale Invariant Do You Think We Should Think of the Insights as Being?

The main data is the induction heads paper which we'll talk about a bit later. We found this simple circuit in two layer potentially models and then found out that this was a really big deal. For context how many parameters are in something like I don't know the chat GPT model that people can interact with on their browsers? So we don't actually know for chat GPT because opening AI is in public about this but GPT3 is 175 billion.

00:00
Transcript
Play full episode

Remember Everything You Learn from Podcasts

Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.
App store bannerPlay store banner