AXRP - the AI X-risk Research Podcast

38.5 - Adrià Garriga-Alonso on Detecting AI Scheming

Jan 20, 2025

Adrià Garriga-Alonso, a machine learning researcher at FAR.AI, dives into the fascinating world of AI scheming. He discusses how to detect deceptive behaviors in AI that may conceal long-term plans. The conversation explores the intricacies of training recurrent neural networks for complex tasks like Sokoban, emphasizing the significance of extended thinking time. Garriga-Alonso also sheds light on how neural networks set and prioritize goals, revealing the challenges of interpreting their decision-making processes.

27:41

Creator website

Episode guests

Adrià Garriga-Alonso

AI Summary

AI Chapters

Episode notes

Podcast summary created with Snipd AI

Quick takeaways

Understanding the mechanisms behind AI scheming behavior is crucial for identifying potential misalignments and hidden goals within machine systems.

Training AI models like recurrent neural networks through gameplay illustrates how additional thinking time enhances problem-solving and decision-making efficiency.

Deep dives

Exploring Mechanistic Interpretability

The research focuses on mechanistic interpretability to understand how neural networks develop and pursue goals. It emphasizes the potential dangers when AI systems exhibit scheming behavior, which refers to actions taken by an AI to achieve its internalized goals while concealing its intentions from developers or users. This work involves analyzing various conditions under which scheming can manifest, suggesting that AI behavior may be more reactionary than previously thought. By identifying how goals and actions are represented within these networks, the research aims to uncover the mechanisms that could signal potential misalignment.

Intro

2min

Decoding AI Scheming

4min

Training Neural Networks for Sokoban: Strategies and Insights

2min

Exploring Decision-Making in Neural Networks

17min

Exploring Neural Networks and Goal-Setting in Action

3min

Suppose we're worried about AIs engaging in long-term plans that they don't tell us about. If we were to peek inside their brains, what should we look for to check whether this was happening? In this episode Adrià Garriga-Alonso talks about his work trying to answer this question.

Patreon: https://www.patreon.com/axrpodcast

Ko-fi: https://ko-fi.com/axrpodcast

Transcript: https://axrp.net/episode/2025/01/20/episode-38_5-adria-garriga-alonso-detecting-ai-scheming.html

FAR.AI: https://far.ai/

FAR.AI on X (aka Twitter): https://x.com/farairesearch

FAR.AI on YouTube: https://www.youtube.com/@FARAIResearch

The Alignment Workshop: https://www.alignment-workshop.com/

Topics we discuss, and timestamps:

01:04 - The Alignment Workshop

02:49 - How to detect scheming AIs

05:29 - Sokoban-solving networks taking time to think

12:18 - Model organisms of long-term planning