Activation Oracles for Interpretability

Our 230th episode with a summary and discussion of last week's big AI news!

Recorded on 01/02/2026

Hosted by Andrey Kurenkov and Jeremie Harris

Feel free to email us your questions and feedback at contact@lastweekinai.com and/or hello@gladstone.ai

Read out our text newsletter and comment on the podcast at https://lastweekin.ai/

In this episode:

• Nvidia's acquisition of AI chip startup Groq for $20 billion highlights a strategic move for enhanced inference technology in GPUs.
• New York's RAISE Act legislation aims to regulate AI safety, marking the second major AI safety bill in the US.
• The launch of GLM 4.7 by Zhipu AI marks a significant advancement in open-source AI models for coding.
• Evaluation of long-horizon AI agents raises concerns about the rising costs and efficiency of AI in performing extended tasks.

Timestamps:

• (00:00:10) Intro / Banter
• (00:01:58) 2025 Retrospective

• Tools & Apps
• (00:24:39) OpenAI bets big on audio as Silicon Valley declares war on screens | TechCrunch

• Applications & Business
• (00:26:39) Nvidia buying AI chip startup Groq for about $20 billion, biggest deal
• (00:34:28) Exclusive | Meta Buys AI Startup Manus, Adding Millions of Paying Users - WSJ
• (00:38:05) Cursor continues acquisition spree with Graphite deal | TechCrunch
• (00:39:15) Micron Hikes CapEx to $20B with 2026 HBM Supply Fully Booked; HBM4 Ramps 2Q26
• (00:42:06) Chinese fabs are reportedly upgrading older ASML DUV lithography chipmaking machines — secondary channels and independent engineers used to soup up Twinscan NXT series

• Projects & Open Source
• (00:47:52) Z.AI launches GLM-4.7, new SOTA open-source model for coding
• (00:50:11) Evaluating AI’s ability to perform scientific research tasks

• Research & Advancements
• (00:54:32) Large Causal Models from Large Language Models
• (00:57:33) Universally Converging Representations of Matter Across Scientific Foundation Models
• (01:02:11) META-RL INDUCES EXPLORATION IN LANGUAGE AGENTS
• (01:07:16) Are the Costs of AI Agents Also Rising Exponentially?
• (01:11:17) METR eval for Opus 4.5
• (01:16:19) How to game the METR plot

• Policy & Safety
• (01:17:24) New York governor Kathy Hochul signs RAISE Act to regulate AI safety | TechCrunch
• (01:20:40) Activation Oracles: Training and Evaluating LLMs as General-Purpose Activation Explainers
• (01:26:46) Monitoring Monitorability
• (01:32:07) Sam Altman is hiring someone to worry about the dangers of AI | The Verge
• (01:33:38) X users asking Grok to put this girl in bikini, Grok is happy obliging - India Today

See Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.