The Gradient: Perspectives on AI cover image

Hattie Zhou: Lottery Tickets and Algorithmic Reasoning in LLMs

The Gradient: Perspectives on AI

00:00

Is There a Difference Between a Weight Change and a Mask Search?

With sufficiently over-parameterized models, you can always find a sub network that works well. It's also potentially interesting, I don't think this has been established, but potentially interesting way of providing regularization to the training process. If your training process was this mask search instead of training the weights, it could provide some interesting generalization properties. That sounds really exciting to me too.

Transcript
Play full episode

The AI-powered Podcast Player

Save insights by tapping your headphones, chat with episodes, discover the best highlights - and more!
App store bannerPlay store banner
Get the app