3min chapter

Clearer Thinking with Spencer Greenberg cover image

What, if anything, do AIs understand? (with ChatGPT Co-Creator Ilya Sutskever)

Clearer Thinking with Spencer Greenberg

CHAPTER

The Importance of Generalization in Neural Networks

The training procedure actually puts in a preference for some of those parameter values over others. So even though all those parameters technically exist, it's sort of more constrained than it seems. One thing that surprises me when I'm using GP3 sometimes is that it seems to know exact texts in some cases. It touches on the question of where does memorization end and generalization begin? And I think they go hand in hand to a significant degree.

00:00

Get the Snipd
podcast app

Unlock the knowledge in podcasts with the podcast player of the future.
App store bannerPlay store banner

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode

Save any
moment

Hear something you like? Tap your headphones to save it with AI-generated key takeaways

Share
& Export

Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode