AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
3min snip
Hattie Zhou: Lottery Tickets and Algorithmic Reasoning in LLMs
The Gradient: Perspectives on AI
Understanding the Lottery Ticket Hypothesis in Neural Networks
The lottery ticket hypothesis states that within a randomly initialized neural network, there exists a sub-network that can achieve the same performance as the full network with all of the weights./nThis could lead to efficiency gains by only having to train fewer weights, saving on training costs and inference costs./nOver-parameterization might be helpful in deep learning because it increases the chances of having a lucky sub-network that is a good starting point for a particular task./nIdentifying lottery tickets can be done through magnitude pruning, where only weights with large magnitudes are kept and trained on./nThis simple method of magnitude pruning raises questions about why it is able to identify these lottery tickets and what makes them good for training.
00:00
Transcript
Episode notes
Get the Snipd
podcast app
Unlock the knowledge in podcasts with the podcast player of the future.
AI-powered
Discover
highlights
Listen to the best highlights from the podcasts you love and dive into the full episode
Save any
moment
Hear something you like? Tap your headphones to save it with AI-generated key takeaways
Share
& Export
Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more
AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
Discover
highlights
Listen to the best highlights from the podcasts you love and dive into the full episode