Episode 20: Hattie Zhou, Mila, on supermasks, iterative learning, and fortuitous forgetting
Generally Intelligent
The Lottery Ticket Hypothesis for Deep Learning
The lottery ticket hypothesis presented a very interesting idea to me, which is that one of the reasons or part of the reason why over-parameterization is so helpful. The way they find these lottery tickets in the original paper by Jonathan Franco and Michael Carbon is by doing magnitude pruning. In fact, I think that even showed that you can do better than the original network by training the sparse network, which is pretty counterintuitive.
00:00
Transcript
Play full episode
Remember Everything You Learn from Podcasts
Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.
