Episode 6: Stochastic Parrot Galactica, November 23, 2022

Emily and Alex discuss MetaAI's bullshit science paper generator, Galactica, along with its defenders. Plus, where could AI actually help scientific research? And more Fresh AI Hell.

Watch the video of this episode on PeerTube.

References:

Imre Lakatos on research programs

Shah, Chirag and Emily M. Bender. 2022. Situating Search. Proceedings of the 2022 ACM SIGIR Conference on Human Information Interaction and Retrieval (CHIIR ’22).

UW RAISE (Responsibility in AI Systems and Experiences)

Stochastic Parrots:
Bender, Emily M., Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021. On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜. In Proceedings of FAccT 2021, pp.610-623.

The Octopus Paper:
Bender Emily M. and Alexander Koller. 2020. Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data. ACL 2020

Palestinian man arrested because of bad machine translation.

Katherine McKittrick, Dear Science and Other Stories

The Sokal Hoax

Safiya Noble, Algorithms of Oppression

Latanya Sweeney, "Discrimination in Online Ad Delivery"

Mehtab Khan and Alex Hanna, The Subjects and Stages of AI Dataset Development: A Framework for Dataset Accountability

(What is 'sealioning'?)
http://wondermark.com/1k62/

Grover:
Raji, Inioluwa Deborah, Emily M. Bender, Amandalynne Paullada, Emily Denton and Alex Hanna. 2021. AI and the Everything in the Whole Wide World Benchmark. Proceedings of the 35th Conference on Neural Information Processing Systems (NeurIPS 2021) Track on Datasets and Benchmarks.

Ben Dickson's coverage of Grover:
Why we must rethink AI benchmarks, Ben Dickson, TechTalks, De

Check out future streams on Twitch. Meanwhile, send us any AI Hell you see.

Our book, 'The AI Con,' is out now! Get your copy now.

Subscribe to our newsletter via Buttondown.

Follow us!

Emily

• Bluesky: emilymbender.bsky.social
• Mastodon: dair-community.social/@EmilyMBender

Alex

• Bluesky: alexhanna.bsky.social
• Mastodon: dair-community.social/@alex
• Twitter: @alexhanna

Music by Toby Menon.
Artwork by Naomi Pleasure-Park.
Production by Christie Taylor.