AXRP - the AI X-risk Research Podcast cover image

19 - Mechanistic Interpretability with Neel Nanda

AXRP - the AI X-risk Research Podcast

CHAPTER

Using a 113 Arithmetic Module You're Learning the Sign Function on 113 Data

The embedding where it's 113 because you want the thing you're taking the module self to be prime for various reasons doesn't make things a bit cleaner. and so there are these like nine possible terms and every neuron the cluster can be well approximated as a linear combination of these. If you delete everything that's not covered by these the model does in bad as well. Each neuron cluster is rank nine even though there's often kind of a hundred or more neurons into cluster which is kind of wild.

00:00
Transcript
Play full episode

Remember Everything You Learn from Podcasts

Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.
App store bannerPlay store banner