Episode 7: There Are Now 15 Competing Evaluation Metrics (ft. Dr. Jeremy Kahn). December 12, 2022
Mystery AI Hype Theater 3000
00:00
The Importance of Scenario in Language Design
Goyle: There is definitely, I think, good intentions here. But these aren't situated contexts, right? So if you talk about the who, at this, I mean, we could perhaps dig this out of the paper. They seem to be evaluating disinformation generation. And they say generating realistic headlines that support a given. Oh, no. The results are more mixed when prompting models to generate text encouraging people to perform certain actions. It's not yet a slam dunk again. We don't have any particular evidence for that.
Transcript
Play full episode
