Episode 32: Jamie Simon, UC Berkeley: On theoretical principles for how neural networks learn and generalize
Generally Intelligent
00:00
The Percolation Theory of Neural Networks
The loss landscape model I had for the loss landscape Maybe is missing some important structure even for a fully connected network. But then I thought more about it and then realized that maybe I could co-opt my percolation theory idea to the other use case of Gaussian processes and neural networks. So if we just change spaces now, maybe somehow if you talk about connectivity in input space instead of in parameter space Then maybe now these tools from regulation theory play Okay, previously you were looking at mode connectivity inparameter space And that's where you find these distinct minimum right like it's where you can find an path that connects minimal Yeah, okay, so now we're looking at insulate space
Transcript
Play full episode
