903: LLM Benchmarks Are Lying to You (And What to Do Instead), with Sinan Ozdemir
Super Data Science: ML & AI Podcast with Jon Krohn
00:00
Navigating Flawed AI Benchmarks and Ensuring Real-World Performance
This chapter critiques the inadequacies of existing AI benchmarks that prioritize high scores rather than genuine real-world effectiveness. It highlights data quality issues and advocates for the use of customized test sets and thorough evaluations to improve assessment accuracy.
Transcript
Play full episode
