4min chapter

Sean Carroll's Mindscape: Science, Society, Philosophy, Culture, Arts, and Ideas cover image

230 | Raphaël Millière on How Artificial Intelligence Thinks

Sean Carroll's Mindscape: Science, Society, Philosophy, Culture, Arts, and Ideas

CHAPTER

Is There a Form of Compositional Representation in These Models?

There is emerging evidence from this line of work that shows that indeed, there might be a form of compositional representation in these models. That is not classical. It does not involve this discrete constituent structure where you can take the representation for dog, mine and biting the biting relation and literally just concatenate these into a complex representation. But it's doing a more sophisticated form of composition that can still keep certain attributes of the different words neatly separated by reading and writing information into distinct subspaces. So again, if you think of the idea that the model tracks the fact that one of the words is the verb, and that the verb relates to the subject in subsets, you might want

00:00

Get the Snipd
podcast app

Unlock the knowledge in podcasts with the podcast player of the future.
App store bannerPlay store banner

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode

Save any
moment

Hear something you like? Tap your headphones to save it with AI-generated key takeaways

Share
& Export

Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode