Reinforcement Learning, Evals, and Human Roles

How supervised fine-tuning, RLHF and RLAIF use rubrics/unit-tests as verifiers, why humans remain essential, and the idea of the economy as RL environments.

Transcript

chevron_right

Play full episode

chevron_right

Transcript

Episode notes

Brendan Foody is the CEO and co-founder of Mercor, the fastest-growing company in history to go from $1M to $500M in revenue (in just 17 months!). At 22, he is also the youngest American unicorn founder ever. Mercor works with 6 of the Magnificent 7 and all top 5 AI labs to help them hire experts to create evaluations and training data that improve their models. In this conversation, Brendan explains why evals have become the critical bottleneck for AI progress, how he discovered this massive opportunity, and what the future of work might look like in an AI-driven economy.

What you’ll learn:

1. Why evals are becoming the primary bottleneck for AI progress and what this means for AI startups

2. How Mercor grew to $500M revenue in 17 months (fastest in history)

3. Brendan’s meeting with xAI that changed his company’s trajectory

4. Which skills and jobs will remain most valuable as AI continues to advance (hint: jobs with “elastic” demand)

5. Why Brendan believes AGI and superintelligence are not happening anytime soon

6. The three unique core values that drove Mercor’s success

7. How Harvard Lampoon writers are making Claude funnier

—

Brought to you by:

WorkOS—Modern identity platform for B2B SaaS, free up to 1 million MAUs

Jira Product Discovery—Atlassian’s new prioritization and roadmapping tool built for product teams