
19 - Mechanistic Interpretability with Neel Nanda
AXRP - the AI X-risk Research Podcast
Is the Principal Axis of the Models Positive or Negative?
The losses you're just definitely being invariant stuff that doesn't matter yes okay so another question i have about this pca is like i want to know what the principal axes were. So there could be an axis of like if you're low on the axis it's like sorry yeah at the start of training and if you're high on theaxis it's like dream training right when it's lower lossyeah okayYeah i agree with that so i guess in that case you'd predict that one of these is just like generally positive for like all tokens or something uh but this is different it's the pca you have different models right and so some models could have better overall loss not i believe
00:00
Transcript
Play full episode
Remember Everything You Learn from Podcasts
Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.