Episode 32: Jamie Simon, UC Berkeley: On theoretical principles for how neural networks learn and generalize
Generally Intelligent
00:00
The Connection Between Gaussian Processes and Machine Learning
A number of connections between Gaussian processes have been discovered in the last few years, especially related to infinite with neural networks. In practice, you're optimizing the loss with respect to the parameters. But nonetheless, it makes youristic sense that this conjecture might give you the right intuition. The reason why that might make sense is because, well, if you're at a local minimum, there are no directions that improve your loss. It turns out we found that the algorithm people were using to find critical points wasn't finding critical points but another type of point which was also bad.
Transcript
Play full episode
