Episode 20: Hattie Zhou, Mila, on supermasks, iterative learning, and fortuitous forgetting
Generally Intelligent
00:00
Is There a Scalable Compositionality?
The way humans store. new compositions, once it's kind of like we run out of memory is by learning them. Machines can just keep outputting things and so they don't update their own. If you're talking about using existing knowledge, but generalizing it compositionally, that could be something that those require continuing learning. But if I want my model to do science, what it needs to do is to learn new things and kind of like, construct them into new obstructions or like new theories.
Transcript
Play full episode
