AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
4min chapter
#118 – Grant Sanderson: Math, Manim, Neural Networks & Teaching with 3Blue1Brown
Lex Fridman Podcast
Using a Dumb Gradient Decayment Algorithm to Find Really Good Solutions
A dumb gradient descent algorithm is able to find really good solutions. It's not terribly surprising that you have these interesting points that exist when you make your space so high dimensional like gpt three what did it have 175 billion parameters yeah. The reason the gradient descent works well with neural networks and not just you know choose however you want to parameterize this space is that that layered structure lets you decompose the derivative in a way that makes it computationally feasible.
00:00
Transcript
Episode notes
Get the Snipd
podcast app
Unlock the knowledge in podcasts with the podcast player of the future.
AI-powered
Discover
highlights
Listen to the best highlights from the podcasts you love and dive into the full episode
Save any
moment
Hear something you like? Tap your headphones to save it with AI-generated key takeaways
Share
& Export
Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more
AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
Discover
highlights
Listen to the best highlights from the podcasts you love and dive into the full episode