Episode 32: Jamie Simon, UC Berkeley: On theoretical principles for how neural networks learn and generalize
Generally Intelligent
00:00
The Conservation of Learnability
The eigenfunction is a function such that similar points according to the krill are assigned similar function values. For most common kernels including neural tanger kernels more slowly varying functions have higher eigenvalue for example if you're on some domain like a sphere or something usually for rotation very kernel. This has been a big discovery of the machine learning theory community over the last few years how to understand the sort of simplicity bias of kernel methods towards high eigenvalue eigenfunctions which happened to be a smooth and then you might actually be like right more often than not because your data is few.
Transcript
Play full episode
