The Risks of Chatbots

3min Snip

00:00

Play full episode

Summary

Transcript

Episode notes

The idea with the phrase stochastic parrots was to basically just give a cute metaphor that would allow people to get a better sense of what this technology is doing. We're drawing here really on the English verb to parrot, which is to repeat back without any understanding and remaining agnostic about the extent to which parrots have internal lives or know what's gonna happen when they say certain things. And so let's leave the actual parrots out of this. So stochastic means randomly, but according to a probability distribution. And parrot here is to parrot to say something about understanding.

Paris Marx is joined by Emily M. Bender to discuss what it means to say that ChatGPT is a “stochastic parrot,” why Elon Musk is calling to pause AI development, and how the tech industry uses language to trick us into buying its narratives about technology.

Emily M. Bender is a professor in the Department of Linguistics at the University of Washington and the Faculty Director of the Computational Linguistics Master’s Program. She’s also the director of the Computational Linguistics Laboratory. Follow Emily on Twitter at @emilymbender or on Mastodon at @emilymbender@dair-community.social.

Tech Won’t Save Us offers a critical perspective on tech, its worldview, and wider society with the goal of inspiring people to demand better tech and a better world. Follow the podcast (@techwontsaveus) and host Paris Marx (@parismarx) on Twitter, and support the show on Patreon.

The podcast is produced by Eric Wickham and part of the Harbinger Media Network.

Also mentioned in this episode:

Emily was one of the co-authors on the “On the Dangers of Stochastic Parrots” paper and co-wrote the “Octopus Paper” with Alexander Koller. She was also recently profiled in New York Magazine and has written about why policymakers shouldn’t fall for the AI hype.
The Future of Life Institute put out the “Pause Giant AI Experiments” letter and the authors of the “Stochastic Parrots” paper responded through DAIR Institute.
Zachary Loeb has written about Joseph Weizenbaum and the ELIZA chatbot.
Leslie Kay Jones has researched how Black women use and experience social media.
As generative AI is rolled out, many tech companies are firing their AI ethics teams.
Emily points to Algorithmic Justice League and AI Incident Database.
Deborah Raji wrote about data and systemic racism for MIT Tech Review.
Books mentioned: Weapons of Math Destruction by Cathy O'Neil, Algorithms of Oppression by Safiya Noble, The Age of Surveillance Capitalism by Shoshana Zuboff, Race After Technology by Ruha Benjamin, Ghost Work by Mary L Gray & Siddharth Suri, Artificial Unintelligence by Meredith Broussard, Design Justice by Sasha Costanza-Chock, Data Conscience: Algorithmic S1ege on our Hum4n1ty by Brandeis Marshall.