Navigating AI Model Benchmarks

In this episode, I spoke with Matt Carey, founding AI engineer at StackOne, founder of AI Demo Days and member of the OpenUK AI Advisory Board.

Everyone needs a friend who works in AI to help them filter the AI news and get the signals from the noise. Matt is that friend for me!

We discussed AI agents, MCP, and the challenges of AI benchmarks, which help explain the disconnect between the benchmark results and the anecdotal experiences of AI users, such as myself.

Links from the episode:

• Google's whitepaper on AI agents
• Anthropic Building Effective AI Agents
• Simon Willison on X
• Thorsten Ball's Joy & Curiosity newsletter
• AI Demo Days
• MCP has a prompt injection problem

Opening theme song:
Cheery Monday by Kevin MacLeod
Link: https://incompetech.filmmusic.io/song/3495-cheery-monday
License: http://creativecommons.org/licenses/by/4.0