Lenny's Podcast: Product | Career | Growth

Why experts writing AI evals is creating the fastest-growing companies in history | Brendan Foody (CEO of Mercor)

274 snips
Sep 18, 2025
Brendan Foody, CEO and co-founder of Mercor, discusses his company's meteoric rise from $1M to $500M in just 17 months, making him the youngest American unicorn founder at 22. He reveals how AI evals have become a critical bottleneck for innovation, and the unique skills that will remain in demand as AI evolves. Brendan shares insights from his pivotal meeting with xAI, the core values that drive Mercor's success, and his skepticism about the timeline for general AI. Get ready for a deep dive into the future of work in an AI-driven economy!
Ask episode
AI Snips
Chapters
Books
Transcript
Episode notes
00:00 / 00:00

Evals Are The Model's Product Requirements

  • If the model is the product, evals are the product requirements that define success and guide improvement.
  • Researchers run experiments against eval sets and use them to direct reinforcement learning to increase capabilities.
00:00 / 00:00

Measure Your Value Chain With Tests

  • Build systematic tests that measure how well AI automates your company's core value chain.
  • Use those tests as PRDs for engineering and as sales collateral demonstrating capability.
00:00 / 00:00

Human Data Market Migrated To Experts

  • The human data market shifted from low-skill crowdsourcing to sourcing high-caliber experts for evals.
  • Labs now need experienced professionals (doctors, lawyers, engineers) to define and measure model capabilities.
Get the Snipd Podcast app to discover more snips from this episode
Get the app