When ChatGPT performed well and where it failed

1131. This week, we talk with ‘Science' magazine senior writer Abigail Eisenstadt about her team's year-long experiment testing ChatGPT's ability to summarize research papers. We look at their methodology, the limitations they realized, and their main finding: that AI could “transcribe” scientific studies but failed to “translate” them with context. 

Read the report: https://www.science.org/do/10.5555/page.2385668/full/chatgpt_project_report_final.pdf

🔗 Share your familect recording in a WhatsApp chat or at 833-214-4475.

🔗 Watch my LinkedIn Learning writing courses.

🔗 Subscribe to the newsletter.

🔗 Take our advertising survey. 

🔗 Get the edited transcript.

🔗 Get Grammar Girl books. 

🔗 Join Grammarpalooza. Get ad-free and bonus episodes at Apple Podcasts or Subtext. Learn more about the difference. 

| HOST: Mignon Fogarty

| VOICEMAIL: 833-214-GIRL (833-214-4475).

| Grammar Girl is part of the Quick and Dirty Tips podcast network.

• Audio Engineer: Dan Feierabend
• Director of Podcast: Holly Hutchings
• Advertising Operations Specialist: Morgan Christianson
• Marketing and Video: Nat Hoopes, Rebekah Sebastian

| Theme music by Catherine Rannus.

| Grammar Girl Social Media: YouTube. TikTok. Facebook.Threads. Instagram. LinkedIn. Mastodon. Bluesky.

Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.