AXRP - the AI X-risk Research Podcast cover image

19 - Mechanistic Interpretability with Neel Nanda

AXRP - the AI X-risk Research Podcast

CHAPTER

Using MLPs to Reverse Enter the Network

A lot of the mental moves that I'm making when I'm trying to reverse enter the network are around discovering thing that things are localized within the model. Most other things do not matter for this input and can be safely ignored. The next thing that matters could be computed solely from the heads or neurons that I identified earlier that are important. There's a core problem called superposition which probably not going to talk about that much later but in brief models want to represent features as directions in space so transformers have these MLP layers where there's a linear map followed by a activation function called a jello.

00:00
Transcript
Play full episode

Remember Everything You Learn from Podcasts

Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.
App store bannerPlay store banner