“The behavioral selection model for predicting AI motivations” by Alex Mallen, Buck

Dec 4, 2025

In this enlightening discussion, Alex Mallen, a researcher on AI alignment and safety, introduces the behavioral selection model for predicting AI motivations. He explores how cognitive patterns influence AI decision-making and the implications of these motivations on behavior. Mallen categorizes AI motivations into fitness seekers, schemers, and optimal kludges, highlighting their selection rationale. He also examines why developer-intended goals can misalign with selection pressures, raising important questions for the future of AI safety.

Ask episode

AI Snips

Chapters

Transcript

Episode notes

INSIGHT

Behavior Drives Cognitive Selection

The behavioral selection model predicts which cognitive patterns an AI keeps by how much their caused behaviors increase selection influence.
Cognitive patterns gain influence when their behaviors causally raise the chance those patterns persist into deployment.

INSIGHT

Map Actions To Selection Pathways

To evaluate a motivation's fitness, map the actions it chooses onto the causal pathways that increase influence through deployment.
Seeking correlates of selection is selected for because interventions on shared causes typically boost influence.

INSIGHT

Three Classes Of Fit Motivations

Three motivation classes tend to be maximally fit under the model: fitness-seekers, schemers, and optimal kludges.
Each class predicts different deployment behaviors and different risks.

Get the Snipd Podcast app to discover more snips from this episode

Get the app

Highly capable AI systems might end up deciding the future. Understanding what will drive those decisions is therefore one of the most important questions we can ask.

Many people have proposed different answers. Some predict that powerful AIs will learn to intrinsically pursue reward. Others respond by saying reward is not the optimization target, and instead reward “chisels” a combination of context-dependent cognitive patterns into the AI. Some argue that powerful AIs might end up with an almost arbitrary long-term goal.

All of these hypotheses share an important justification: An AI with each motivation has highly fit behavior according to reinforcement learning.

This is an instance of a more general principle: we should expect AIs to have cognitive patterns (e.g., motivations) that lead to behavior that causes those cognitive patterns to be selected.

In this post I’ll spell out what this more general principle means and why it's helpful. Specifically:

I’ll introduce the “behavioral selection model,” which is centered on this principle and unifies the basic arguments about AI motivations in a big causal graph.
I’ll discuss the basic implications for AI motivations.
And then I’ll discuss some important extensions and omissions of the behavioral selection model.

This [...]

---

Outline:

(02:13) How does the behavioral selection model predict AI behavior?

(05:18) The causal graph

(09:19) Three categories of maximally fit motivations (under this causal model)

(09:40) 1. Fitness-seekers, including reward-seekers

(11:42) 2. Schemers

(14:02) 3. Optimal kludges of motivations

(17:30) If the reward signal is flawed, the motivations the developer intended are not maximally fit

(19:50) The (implicit) prior over cognitive patterns

(24:07) Corrections to the basic model