Can We Decode the Internal Encodings of a Language Model?

4min Snip

00:00

Play full episode

Summary

Transcript

Episode notes

Research is going beyond looking at the behavior of these models, and actually trying to make some stronger claims about the internal mechanisms that explain their behavior. That's a whole area of research that has come to be known as mechanistic interpretability in computer science and computational linguistics. We can take a leaf from the book of decades of research in neuroscience and psychology to try to understand the mechanisms of the black boxes that are the human mind or the human brain. The most refined kind of approach would be to look at toy models that are very, very small in terms of the number of parameters they have.

Welcome to another episode of Sean Carroll's Mindscape. Today, we're joined by Raphaël Millière, a philosopher and cognitive scientist at Columbia University. We'll be exploring the fascinating topic of how artificial intelligence thinks and processes information. As AI becomes increasingly prevalent in our daily lives, it's important to understand the mechanisms behind its decision-making processes. What are the algorithms and models that underpin AI, and how do they differ from human thought processes? How do machines learn from data, and what are the limitations of this learning? These are just some of the questions we'll be exploring in this episode. Raphaël will be sharing insights from his work in cognitive science, and discussing the latest developments in this rapidly evolving field. So join us as we dive into the mind of artificial intelligence and explore how it thinks.

[The above introduction was artificially generated by ChatGPT.]

Support Mindscape on Patreon.

Raphaël Millière received a DPhil in philosophy from the University of Oxford. He is currently a Presidential Scholar in Society and Neuroscience at the Center for Science and Society, and a Lecturer in the Philosophy Department at Columbia University. He also writes and organizes events aimed at a broader audience, including a recent workshop on The Challenge of Compositionality for Artificial Intelligence.

See Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.