19 - Mechanistic Interpretability with Neel Nanda
AXRP - the AI X-risk Research Podcast
00:00
Using Contextual Information in Model Composition
Small dimension well subspace yeah yes small small dimensional you can kind of think of this as the residual stream let's say a thousand dimensional vector and we just cut off the first 50 ignore everything else yep but the network can choose whatever coordinate system involves. This means that the head can choose different subspaces of the destination or source residual streams to pick up on if it so chooses This lets the model kind of choose different bits to compose with and this matters because one of the things that mechanistic determinability tries hard to exploit is the fact that model computation often seems kind of sparse where there are only some masses.
Transcript
Play full episode
