Episode 32: Jamie Simon, UC Berkeley: On theoretical principles for how neural networks learn and generalize
Generally Intelligent
00:00
The Importance of Mode Connectivity in Deep Learning
I think there's no first order reason why I Don't know about moto connectivity for input data like input data to output. It matters because it tells us about the structure of the minimizers of the neural network and mode connectivity has remained a sort of mains day objective interest in optimization studies of deep neural nets. The problem space is simple But if early in training if you sort of branch off your optimization early in training then the two solutions are connected by as simple a path As the women otherwise which tells you you ended up in a different region.
Transcript
Play full episode
