Episode 20: Hattie Zhou, Mila, on supermasks, iterative learning, and fortuitous forgetting
Generally Intelligent
00:00
Is Knowledge Evolution Really Useful in Transfer Learning?
This method isn't particularly competitive to other things you can do, such as pre-training. But there are still cases where this could be helpful. So it could be going back to the information that's stored in large pre-trained models. And Wi-Fi and tuning helps to get certain, to extract certain capabilities from it.
Transcript
Play full episode
