Episode 32: Jamie Simon, UC Berkeley: On theoretical principles for how neural networks learn and generalize
Generally Intelligent
00:00
The Four Hidden Layer Network at Infinite Width
In theory like we should be able to take any fully connected network and collapse it into a single layer network Yeah, okay, so there's one thing missing Yes. There was no advantage to depth It wasn't actually why are you anything? And this is the sort of result that I really like because it is like a very clear signal. Even though even though every input vector Has to have the same norm as our other input vector Although enforcing that though doesn't really affect performance much We still see good agreement.
Transcript
Play full episode
