5min chapter

Sean Carroll's Mindscape: Science, Society, Philosophy, Culture, Arts, and Ideas cover image

230 | Raphaël Millière on How Artificial Intelligence Thinks

Sean Carroll's Mindscape: Science, Society, Philosophy, Culture, Arts, and Ideas

CHAPTER

The Connection Between Machine Learning and Compression

Chat GPT and language models generally can be thought of as a kind of blurry JPEG of the web, where they get trained to compress the web. And then at inference time, when you're generating a text with these models, it's a form of lossy decompression that involves interpolation in the same way. So there is I think it's a very interesting test case for intuitions, because I think this this metaphor, this analogy, parts of it are pumping the right intuitions.

00:00

Get the Snipd
podcast app

Unlock the knowledge in podcasts with the podcast player of the future.
App store bannerPlay store banner

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode

Save any
moment

Hear something you like? Tap your headphones to save it with AI-generated key takeaways

Share
& Export

Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode