The Importance of Generalization in Neural Networks

Iliya: I think there is this blog post on your sub stack that said something like you just weren't going to rely on generalization at all. The new question is like well what if we actually did understand it yeah, he says. "What if we can actually say something meaningful about generalization and I think that's like a really good question" He continues with the idea of splitting datasets existing datasets into easy problems and harder problems where the easy problems are defined as what a small model gets right. And then they're trying to improve or improve how can we get how can we improved the accuracy of a large model on the whole dataset?

Transcript

Play full episode

Transcript

Episode notes

Remember Everything You Learn from Podcasts

Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.

Get the app