S3E6: Geoffrey Hinton on the Exploration of Thought
Theory and Practice
00:00
How the Neural Networks Work?
Starcastic gradient sent really does work on big data sets, and it can do amazing things. I think we'll get even more progress by getting better ideas like using fast weights that are adapted a different time scale from the standard weights. We're seeing emergence of an evolutionary dynamic where some architectures are surviving because they can take advantage of this dense arithmetic. i don't think we would have neural networks in the form that we do to day without specialized hardware that can do extremely fast matrix multiplies. Every computer will be somewhat differently made - just like biological computers or plants.
Transcript
Play full episode
