Episode 20: Hattie Zhou, Mila, on supermasks, iterative learning, and fortuitous forgetting
Generally Intelligent
00:00
Increasing Compositionality Through Iterative Learning
There have been some papers that take this idea and use it in neural networks. And they did show that you can increase compositionality through this. So if you keep doing this process iteratively, you get agents that start talking in a language that's compositional. But I'm not sure that that is enough. It's not necessarily in the case that that compositionality should be preserved or that it's always the case that you learn compositional things first.
Transcript
Play full episode
