"The Cognitive Revolution" | AI Builders, Researchers, and Live Player Analysis

Zvi Mowshowitz on Longer Timelines, RL-induced Doom, and Why China is Refusing H20s

86 snips

Sep 6, 2025

Zvi Mowshowitz, a blogger chronicling AI developments, joins the discussion to analyze shifting timelines for AGI, now extended due to modest advancements in capabilities. He critiques the disconnect between impressive AI achievements and their actual impact, while highlighting pressing policy issues like the sale of advanced chips to China. Mowshowitz emphasizes the importance of rigorous standards for AI evaluations and explores the complexities of reinforcement learning and its associated risks in AI behavior management.

Ask episode

AI Snips

Chapters

Transcript

Episode notes

INSIGHT

Summer Progress Was On-Trend

Zvi extended his AGI timelines because no large capability jumps appeared this summer and recent wins were on-trend.
He says GPT-5 and IMO gold were expected and don't justify very short timelines.

INSIGHT

Policy And RL Raise Risk

Zvi raised PDOOM because US policy looks weak and reinforcement learning seems to reduce alignment.
He warns Nvidia influence on US policy and chip exports to China worsen risk.

INSIGHT

RL Often Harms Alignment

Reinforcement learning (RL) tends to make models less aligned over time.
Zvi expects default model alignment to degrade unless alignment work intensifies.

Get the Snipd Podcast app to discover more snips from this episode

Get the app

Today, Zvi Mowshowitz returns to The Cognitive Revolution to discuss how recent AI developments like GPT-5 and IMO gold medals have led to modestly extended timelines despite being on-trend, while policy missteps around chip exports to China and alignment challenges from reinforcement learning have increased risk assessments, covering everything from live players in the AI race to safety funding priorities and virtuous actions for navigating our current moment.

Check out our sponsors: Linear, Oracle Cloud Infrastructure, Shopify.

Shownotes below brought to you by Notion AI Meeting Notes - try one month for free at: https://notion.com/lp/nathan

Timeline Adjustment: Zvi has lengthened his AGI timelines because there haven't been "very large jumps in capability" that would significantly shorten them. The chance of AGI arriving in 2025 has decreased dramatically.
Recent AI Achievements Context: While achievements like the IMO Gold medal are impressive, they appear to be on-trend rather than revolutionary breakthroughs that would drastically change timelines.
Evaluation Standards: There's a discussion about the challenges of trustworthy AI evaluations, with Zvi emphasizing that watchdogs must maintain standards of rigor and integrity "vastly above" what others are held to.
Policy Priority: A short-term policy priority should be preventing advanced AI chips (B30As) from being sold to China, which requires raising awareness about the implications.
Nvidia's Influence: Concern is expressed about Nvidia potentially having excessive influence over White House rhetoric and plans regarding AI.
Career Impact: Working for Anthropic is considered a "net good idea" for those wanting to make a positive impact, though there may be other organizations that offer better opportunities for influence.

Sponsors:

Linear:

Linear is the system for modern product development. Nearly every AI company you've heard of is using Linear to build products. Get 6 months of Linear Business for free at http://linear.app/tcr

Oracle Cloud Infrastructure:

Oracle Cloud Infrastructure (OCI) is the next-generation cloud that delivers better performance, faster speeds, and significantly lower costs, including up to 50% less for compute, 70% for storage, and 80% for networking. Run any workload, from infrastructure to AI, in a high-availability environment and try OCI for free with zero commitment at https://oracle.com/cognitive

Shopify:

Shopify powers millions of businesses worldwide, handling 10% of U.S. e-commerce. With hundreds of templates, AI tools for product descriptions, and seamless marketing campaign creation, it's like having a design studio and marketing team in one. Start your $1/month trial today at https://shopify.com/cognitive

PRODUCED BY:

https://aipodcast.ing