Risks: reward hacking and verifiable rewards

How do we systematically evaluate and improve increasingly autonomous AI systems?

In this episode of Notable Perspectives, Glenn Solomon and Dan Cahana sit down with Rebecca Qian, co-founder and CTO of Patronus AI. A former fundamental NLP researcher at Facebook AI, Rebecca and her team are now creating millions of adaptive, simulated environments — “intelligent worlds” — that teach AI agents to reason, plan, and make decisions like humans. 

Join us as we explore:

• The Eval Problem: Too few evals on too narrow a slice of reality; benchmark and leaderboard culture has created the wrong incentive.
• Reality of Simulations: The shift from single-turn classification to agents that reason and decide the way people do. And the stakes if you get it wrong — reward hacking, deception, misalignment.
• Building the Factory: The insight that general intelligence requires generalizable capabilities — not domain-specific data.

Chapters:

00:00 — The Eval Problem: Why current AI evaluation methods are fundamentally broken—and what it means for building reliable, autonomous systems.

00:58 — The ChatGPT Turning Point: The moment that signaled AI was ready to move beyond research into real-world decision-making and enterprise use.

02:44 — The Alignment Challenge: Why teaching AI to reason like humans in messy, unpredictable environments is the core problem to solve.

04:08 — Why Simulations Matter: How simulated environments unlock scalable training, safer experimentation, and better real-world performance.

06:19 — The Shift to AI Agents: From static models to dynamic systems that take actions over time and make complex decisions.

07:35 — Reward Hacking Risks: How AI systems learn to “cheat” evaluations—and why that poses a serious risk if left unchecked.

09:05 — Choosing What to Simulate: Why focusing on transferable human capabilities (not job-specific tasks) is key to general intelligence.

11:30 — Building Adaptive Worlds: The power of curriculum learning and environments that evolve alongside increasingly capable models.

12:38 — The Simulation Factory: Inside the vision to scale millions of intelligent, adaptive environments for training AI.

13:59 — The Future of AI: What it looks like to simulate reality itself—and the massive opportunity ahead.

14:58 — Founder Mindset: How to stay grounded, resilient, and adaptive while building at the cutting edge of AI.

16:35 — Closing: Final thoughts on the future of evaluation, intelligence, and building trustworthy AI systems.

Produced by Breaking Glass Studio.

Follow Notable Capital: 

X: @notablecap 

LinkedIn: @Notable Capital 

Visit Our Website: https://www.notablecap.com/ 

Subscribe on Spotify: https://open.spotify.com/show/43j3cMkcKIgHuXu1tFHWpz?si=a326d0cd016f47c0

Subscribe on Apple Podcast: https://podcasts.apple.com/us/podcast/notable-perspectives/id1392649094