
19 - Mechanistic Interpretability with Neel Nanda
AXRP - the AI X-risk Research Podcast
A Softmax Is a Matrix of Keys and Values and Attention Heads
There's this paper that i read once his title was transformer feedforward layers are key value memories they're basically argued that look we know what uh the mlp layers are doing. A lot of this is just stuff that can be done with a lookup table like in some sense if you want to do an if statement. I've anecdotally heard some claims that some of the results replicate and some don't seem to replicate very well but obviously this is not particularly reliable data. My personal intuition would be that this is part of the story but a pretty limited part and where it's very easy to get some for this to be like a small part of what a neuron is doing.
00:00
Transcript
Play full episode
Remember Everything You Learn from Podcasts
Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.