"The Cognitive Revolution" | AI Builders, Researchers, and Live Player Analysis

Scaling "Thinking": Gemini 2.5 Tech Lead Jack Rae on Reasoning, Long Context, & the Path to AGI

Apr 5, 2025

Jack Rae, Principal Research Scientist at Google DeepMind, shares insights on the revolutionary Gemini 2.5 model. He discusses the surge in efficacy of reasoning techniques and what this means for AI's journey towards artificial general intelligence. Rae dives into the interplay between human data and model behavior, the ethical challenges of reinforcement learning, and the nuances of AI reasoning. Listeners gain a unique perspective on the future of AI, exploring its potential applications and the hurdles that lie ahead in achieving true cognitive capabilities.

01:16:28

Creator website

Episode guests

Jack Rae

AI Summary

AI Chapters

Episode notes

Podcast summary created with Snipd AI

Quick takeaways

The Gemini 2.5 Pro model represents a significant leap in AI's ability to process extensive input contexts, transforming user workflows.

Recent advancements in reinforcement learning techniques highlight their critical role in improving AI model reasoning and decision-making capabilities.

The exploration of reasoning and agency in AI raises vital questions about developing autonomous behaviors while enhancing cognitive processes.

Deep dives

Gemini 2.5 Pro's Milestone in AI Development

Gemini 2.5 Pro represents a significant advancement in AI, marking its status as a top-performing model across various critical dimensions. This model excels in handling extensive input contexts, demonstrating a deep understanding of hundreds of thousands of tokens. The performance improvement has been observable, prompting many users to rethink their workflows and explore its capabilities in exceeding expectations. This upgrade invites ongoing discussion about the trajectory and potential of AI technology in achieving advanced artificial general intelligence (AGI).

Intro

4min

Navigating the Future of AI

5min

Navigating AI Research and Personal Development

4min

Trends in AI Development: Reasoning Models and Reinforcement Learning

6min

Exploring Cognitive Behaviors in AI Models

5min

Model Reasoning: Balancing Language and Transparency

8min

Navigating the Ethics of Reinforcement Learning in AI Reasoning

4min

Navigating AI Reasoning and Interpretability

24min

Exploring Gemini 2.5 Pro: Multimodal Advances and Future Challenges

13min

In this illuminating episode of The Cognitive Revolution, host Nathan Labenz speaks with Jack Rae, principal research scientist at Google DeepMind and technical lead on Google's thinking and inference time scaling work. They explore the technical breakthroughs behind Google's Gemini 2.5 Pro model, discussing why reasoning techniques are suddenly working so effectively across the industry and whether these advances represent true breakthroughs or incremental progress. The conversation delves into critical questions about the relationship between reasoning and agency, the role of human data in shaping model behavior, and the roadmap from current capabilities to AGI, providing listeners with an insider's perspective on the trajectory of AI development.

SPONSORS:

Oracle Cloud Infrastructure (OCI): Oracle Cloud Infrastructure offers next-generation cloud solutions that cut costs and boost performance. With OCI, you can run AI projects and applications faster and more securely for less. New U.S. customers can save 50% on compute, 70% on storage, and 80% on networking by switching to OCI before May 31, 2024. See if you qualify at https://oracle.com/cognitive

Shopify: Shopify is revolutionizing online selling with its market-leading checkout system and robust API ecosystem. Its exclusive library of cutting-edge AI apps empowers e-commerce businesses to thrive in a competitive market. Cognitive Revolution listeners can try Shopify for just $1 per month at https://shopify.com/cognitive

NetSuite: Over 41,000 businesses trust NetSuite by Oracle, the #1 cloud ERP, to future-proof their operations. With a unified platform for accounting, financial management, inventory, and HR, NetSuite provides real-time insights and forecasting to help you make quick, informed decisions. Whether you're earning millions or hundreds of millions, NetSuite empowers you to tackle challenges and seize opportunities. Download the free CFO's guide to AI and machine learning at https://netsuite.com/cognitive

PRODUCED BY:

https://aipodcast.ing

CHAPTERS:

(00:00) About the Episode

(05:09) Introduction and Welcome

(07:28) RL for Reasoning

(10:46) Research Time Management

(13:41) Convergence in Model Development

(18:31) RL on Smaller Models (Part 1)

(20:01) Sponsors: Oracle Cloud Infrastructure (OCI) | Shopify

(22:35) RL on Smaller Models (Part 2)

(23:30) Sculpting Cognitive Behaviors

(25:05) Language Switching Behavior

(28:02) Sharing Chain of Thought

(32:03) RL on Chain of Thought (Part 1)

(33:46) Sponsors: NetSuite

(35:19) RL on Chain of Thought (Part 2)

(35:26) Eliciting Human Reasoning

(39:27) Reasoning vs. Agency

(40:17) Understanding Model Reasoning

(44:29) Reasoning in Latent Space

(47:54) Interpretability Challenges

(51:36) Platonic Model Hypothesis

(56:05) Roadmap to AGI

(01:00:57) Multimodal Integration

(01:04:38) System Card Questions

(01:07:51) Long Context Capabilities

(01:13:49) Outro

Remember Everything You Learn from Podcasts

Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.

"The Cognitive Revolution" | AI Builders, Researchers, and Live Player Analysis

Scaling "Thinking": Gemini 2.5 Tech Lead Jack Rae on Reasoning, Long Context, & the Path to AGI

Episode guests

Podcast summary created with Snipd AI

Quick takeaways

Deep dives

Gemini 2.5 Pro's Milestone in AI Development

Breakthroughs in Reinforcement Learning Techniques

The Role of Human Data in Model Training

Exploring Reasoning and Agency in AI

The Pathway to AGI and Remaining Challenges

Remember Everything You Learn from Podcasts