AXRP - the AI X-risk Research Podcast cover image

19 - Mechanistic Interpretability with Neel Nanda

AXRP - the AI X-risk Research Podcast

CHAPTER

Is There a Suspension in Test Loss?

There's actually this relatively gradual underlying thing of the network that drives the sudden transition so I guess my first question is how is it the case that like this gradual development and this like gradual dying down of the memorization results in a sudden decrease in test loss at some point? sure. yeah on a big picture I think of the story of this paper is hey see this like sudden phenomenon that you thought was sudden there's actually this like relatively gradual underlyingthing of the network  that drive the sudden transition. sorry I didn't quite follow the question so if the underlying driver of like successful generalization is basically like these circuits, we're saying that these circuits are gradually appearing then they die away

00:00
Transcript
Play full episode

Remember Everything You Learn from Podcasts

Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.
App store bannerPlay store banner