AXRP - the AI X-risk Research Podcast cover image

19 - Mechanistic Interpretability with Neel Nanda

AXRP - the AI X-risk Research Podcast

CHAPTER

Is the Principal Axis of the Models Positive or Negative?

The losses you're just definitely being invariant stuff that doesn't matter yes okay so another question i have about this pca is like i want to know what the principal axes were. So there could be an axis of like if you're low on the axis it's like sorry yeah at the start of training and if you're high on theaxis it's like dream training right when it's lower lossyeah okayYeah i agree with that so i guess in that case you'd predict that one of these is just like generally positive for like all tokens or something uh but this is different it's the pca you have different models right and so some models could have better overall loss not i believe

00:00
Transcript
Play full episode

Remember Everything You Learn from Podcasts

Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.
App store bannerPlay store banner