
Episode 20: Hattie Zhou, Mila, on supermasks, iterative learning, and fortuitous forgetting
Generally Intelligent
Is There a Generalization of Desirable Versus Unwanted?
I think there's other parts to it too though, like desirable and undesirable can apply even without goals. And I can imagine this being useful in that setting too. Like we don't necessarily have to be an agent or an explicit goal. If you have a forgetting mechanism that the model has to do, but it can figure out how to do that while it's also learning, could be an interesting general way of solving that. Yeah. You want your model to basically, there's no separate objective function for the forgetting. It's almost like you just have attention and attention learns to do different things.
00:00
Transcript
Play full episode
Remember Everything You Learn from Podcasts
Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.