S3E6: Geoffrey Hinton on the Exploration of Thought
Theory and Practice
How the Neural Networks Work?
Starcastic gradient sent really does work on big data sets, and it can do amazing things. I think we'll get even more progress by getting better ideas like using fast weights that are adapted a different time scale from the standard weights. We're seeing emergence of an evolutionary dynamic where some architectures are surviving because they can take advantage of this dense arithmetic. i don't think we would have neural networks in the form that we do to day without specialized hardware that can do extremely fast matrix multiplies. Every computer will be somewhat differently made - just like biological computers or plants.
00:00
Transcript
Play full episode
Remember Everything You Learn from Podcasts
Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.
