AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
The Importance of Generalization in Neural Networks
The training procedure actually puts in a preference for some of those parameter values over others. So even though all those parameters technically exist, it's sort of more constrained than it seems. One thing that surprises me when I'm using GP3 sometimes is that it seems to know exact texts in some cases. It touches on the question of where does memorization end and generalization begin? And I think they go hand in hand to a significant degree.