Generally Intelligent cover image

Episode 20: Hattie Zhou, Mila, on supermasks, iterative learning, and fortuitous forgetting

Generally Intelligent

00:00

The Lottery Ticket Hypothesis

The lottery ticket hypothesis presented a very interesting idea to me, which is that one of the reasons or part of the reason why over-parameterization is so helpful. So if you have used lucky subnetworks, then the model could just leverage that good initial spot and go down a pretty nice path of learning. The way they find these lottery tickets in the original paper by Jonathan Franco and Michael Carbon is by doing magnitude pruning.

Transcript
Play full episode

The AI-powered Podcast Player

Save insights by tapping your headphones, chat with episodes, discover the best highlights - and more!
App store bannerPlay store banner
Get the app