Generally Intelligent cover image

Episode 20: Hattie Zhou, Mila, on supermasks, iterative learning, and fortuitous forgetting

Generally Intelligent

00:00

The Fortuitous Forgetting Paper

The idea is what if it's not about what the model is learning each time? It's also about what it's losing each time, what information is being forgotten. We came up with this hypothesis that if you can identify a way to forget information that you deem undesirable from a trained model, then you can just slowly steer the model away from things defined undesirable towards more desirable characteristics. And we do see disproportionately, perhaps not surprisingly, that the difficult examples are forgotten more.

Transcript
Play full episode

The AI-powered Podcast Player

Save insights by tapping your headphones, chat with episodes, discover the best highlights - and more!
App store bannerPlay store banner
Get the app