Evaluating AI Models: Foundations for Future Interaction

Sholto Douglas, a Member of Technical Staff at Anthropic, joined Unsupervised Learning to break down why coding is the clearest early signal of model progress, how AI agents are already accelerating research, and what it’ll take to unlock real-world breakthroughs in fields like biology and robotics.

(0:00) Intro
(0:48) Claude 4
(1:30) Capabilities and Improvements
(2:29) Practical Applications and Advice
(3:04) Future of AI in Coding
(4:38) Managing Multiple AI Models
(11:20) The Barrier to Agents is Reliability
(16:35) Agents Conducting Research
(19:54) Impact of Models on World GDP
(25:14) Most Important Metrics in Model Improvement
(29:53) Stories of Model Creativity
(32:45) How Often Will New Models Be Shipped in the Future?
(39:51) Day-to-Day Work of AI Researchers
(46:46) The Future of AI and Society
(51:26) Quickfire

With your co-hosts: 

@jacobeffron 

- Partner at Redpoint, Former PM Flatiron Health 

@patrickachase 

- Partner at Redpoint, Former ML Engineer LinkedIn 

@ericabrescia 

- Former COO Github, Founder Bitnami (acq’d by VMWare) 

@jordan_segall 

- Partner at Redpoint