Mystery AI Hype Theater 3000 cover image

Episode 7: There Are Now 15 Competing Evaluation Metrics (ft. Dr. Jeremy Kahn). December 12, 2022

Mystery AI Hype Theater 3000

CHAPTER

The Importance of Scenario in Language Design

Goyle: There is definitely, I think, good intentions here. But these aren't situated contexts, right? So if you talk about the who, at this, I mean, we could perhaps dig this out of the paper. They seem to be evaluating disinformation generation. And they say generating realistic headlines that support a given. Oh, no. The results are more mixed when prompting models to generate text encouraging people to perform certain actions. It's not yet a slam dunk again. We don't have any particular evidence for that.

00:00
Transcript
Play full episode

Remember Everything You Learn from Podcasts

Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.
App store bannerPlay store banner