Scalable Oversight in AI Safety

Our 209th episode with a summary and discussion of last week's big AI news!
Recorded on 05/16/2025

Hosted by Andrey Kurenkov and Jeremie Harris.
Feel free to email us your questions and feedback at contact@lastweekinai.com and/or hello@gladstone.ai

Read out our text newsletter and comment on the podcast at https://lastweekin.ai/.

Join our Discord here! https://discord.gg/nTyezGSKwP

In this episode:

• OpenAI has decided not to transition from a nonprofit to a for-profit entity, instead opting to become a public benefit corporation influenced by legal and civic discussions.
• Trump administration meetings with Saudi Arabia and the UAE have opened floodgates for AI deals, leading to partnerships with companies like Nvidia and aiming to bolster AI infrastructure in the Middle East.
• DeepMind introduced Alpha Evolve, a new coding agent designed for scientific and algorithmic discovery, showing improvements in automated code generation and efficiency.
• OpenAI pledges greater transparency in AI safety by launching the Safety Evaluations Hub, a platform showcasing various safety test results for their models.

Timestamps + Links:

• (00:00:00) Intro / Banter
• (00:01:41) News Preview
• (00:02:26) Response to listener comments
• Applications & Business
  • (00:03:00) OpenAI says non-profit will remain in control after backlash
  • (00:13:23) Microsoft Moves to Protect Its Turf as OpenAI Turns Into Rival
  • (00:18:07) TSMC’s 2nm Process Said to Witness ‘Unprecedented’ Demand, Exceeding 3nm Due to Interest from Apple, NVIDIA, AMD, & Many Others
  • (00:21:42) NVIDIA’s Global Headquarters Will Be In Taiwan, With CEO Huang Set To Announce Site Next Week, Says Report
  • (00:23:58) CoreWeave in Talks for $1.5 Billion Debt Deal 6 Weeks After IPO
• Tools & Apps
  • (00:26:39) The Day Grok Told Everyone About ‘White Genocide’
  • (00:32:58) Figma releases new AI-powered tools for creating sites, app prototypes, and marketing assets
  • (00:36:12) Google’s bringing Gemini to your car with Android Auto
  • (00:38:49) Google debuts an updated Gemini 2.5 Pro AI model ahead of I/O
  • (00:45:09) Hugging Face releases a free Operator-like agentic AI tool
• Projects & Open Source
  • (00:47:42) Stability AI releases an audio-generating model that can run on smartphones
  • (00:50:47) Freepik releases an ‘open’ AI image generator trained on licensed data
  • (00:54:22) AM-Thinking-v1: Advancing the Frontier of Reasoning at 32B Scale
  • (01:01:29) BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Dataset
• Research & Advancements
  • (01:05:40) DeepMind claims its newest AI tool is a whiz at math and science problems
  • (01:12:31) Absolute Zero: Reinforced Self-play Reasoning with Zero Data
  • (01:19:44) How far can reasoning models scale?
  • (01:26:47) HealthBench: Evaluating Large Language Models Towards Improved Human Health
• Policy & Safety
  • (01:34:10) Trump administration officially rescinds Biden’s AI diffusion rules
  • (01:37:08) Trump’s Mideast Visit Opens Floodgate of AI Deals Led by Nvidia
  • (01:44:04) Scaling Laws For Scalable Oversight
  • (01:49:43) OpenAI pledges to publish AI safety test results more often