Is There a Form of Compositional Representation in These Models?

4min Snip

00:00

Play full episode

Summary

Transcript

Episode notes

There is emerging evidence from this line of work that shows that indeed, there might be a form of compositional representation in these models. That is not classical. It does not involve this discrete constituent structure where you can take the representation for dog, mine and biting the biting relation and literally just concatenate these into a complex representation. But it's doing a more sophisticated form of composition that can still keep certain attributes of the different words neatly separated by reading and writing information into distinct subspaces. So again, if you think of the idea that the model tracks the fact that one of the words is the verb, and that the verb relates to the subject in subsets, you might want

Welcome to another episode of Sean Carroll's Mindscape. Today, we're joined by Raphaël Millière, a philosopher and cognitive scientist at Columbia University. We'll be exploring the fascinating topic of how artificial intelligence thinks and processes information. As AI becomes increasingly prevalent in our daily lives, it's important to understand the mechanisms behind its decision-making processes. What are the algorithms and models that underpin AI, and how do they differ from human thought processes? How do machines learn from data, and what are the limitations of this learning? These are just some of the questions we'll be exploring in this episode. Raphaël will be sharing insights from his work in cognitive science, and discussing the latest developments in this rapidly evolving field. So join us as we dive into the mind of artificial intelligence and explore how it thinks.

[The above introduction was artificially generated by ChatGPT.]

Support Mindscape on Patreon.

Raphaël Millière received a DPhil in philosophy from the University of Oxford. He is currently a Presidential Scholar in Society and Neuroscience at the Center for Science and Society, and a Lecturer in the Philosophy Department at Columbia University. He also writes and organizes events aimed at a broader audience, including a recent workshop on The Challenge of Compositionality for Artificial Intelligence.

See Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.