
Episode 20: Hattie Zhou, Mila, on supermasks, iterative learning, and fortuitous forgetting
Generally Intelligent
Is There a Scalable Compositionality?
The way humans store. new compositions, once it's kind of like we run out of memory is by learning them. Machines can just keep outputting things and so they don't update their own. If you're talking about using existing knowledge, but generalizing it compositionally, that could be something that those require continuing learning. But if I want my model to do science, what it needs to do is to learn new things and kind of like, construct them into new obstructions or like new theories.
00:00
Transcript
Play full episode
Remember Everything You Learn from Podcasts
Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.