AXRP - the AI X-risk Research Podcast

11 - Attainable Utility and Power with Alex Turner

Sep 25, 2021

01:27:36

Creator website

Highlights

AI Chapters

Episode notes

The Top 3 Properties of a Prism

01:36

How to Preserve the Utility of a Reward Function, Right?

01:54

How Does This Fit Into an Alignment?

02:37

Introduction

2min

The Second Hope of Side Effect Research

2min

Is Objective Maximization a Bad Frame?

3min

The Top 3 Properties of a Prism

2min

How to Preserve the Ability to Do Random Things?

2min

Is There a Good Noap Policy?

2min

Is It Easier to Measure the Human Action Space?

5min

The Inaction Policy of Maximizing Power

2min

Power Maintenance vs Power Maximization

2min

10.

The Difference Between Agent and Non Agent Parts of the Environment

4min

11.

What's the Alternative to Roll Outs?

2min

12.

How to Preserve the Utility of a Reward Function, Right?

2min

13.

Aspects of the Utility Maximization Framework

2min

14.

Getting the Right Objective and Loading the Objective Into the Agent

3min

15.

The Relationship Between Corrigeibility and Utility Preservation

4min

16.

Is It Useful to Have an Agent That Preserves Its Ability to Achieve a Wide Range of Things?

2min

17.

The Relationship Between Power and Utility Preservation

3min

18.

Is It the Case That Automa Policies Tend to Seek Power All the Time?

2min

19.

What Kinds of Symmetries Do You Need for Instrumental Conversion?

5min

20.

Is There a Game Over Screen?

1min

21.

Is That How Instrumental Convergence Works?

1min

22.

The Reward Function for an Ai System Isn't Optimistic

2min

23.

How Does This Fit Into an Alignment?

3min

24.

Ike, Is It Motivating the Ai Systems to Have Really Bad Consequences?

2min

25.

The Chain of Arguments, Sir?

2min

26.

Is Human Approval of an Action Based on a I Gaining Some Power?

3min

27.

Is There a Free Lunch Theorem?

3min

28.

Is There a Necessity in the Environment?

2min

29.

Is There a Way to Maximize Your Utility?

3min

30.

A Power Query - What Extensions Are Most Valuable to You?

4min

31.

Is There a Bounded Optimality?

2min

32.

Does Alex Turner Research Taste Look Like?

4min

33.

What's the Alex Turner a Genda?

3min

34.

What's Like a Steelman of the Case Gainst Power Seeking or Against?

3min

Many scary stories about AI involve an AI system deceiving and subjugating humans in order to gain the ability to achieve its goals without us stopping it. This episode's guest, Alex Turner, will tell us about his research analyzing the notions of "attainable utility" and "power" that underlie these stories, so that we can better evaluate how likely they are and how to prevent them.

Topics we discuss:

- Side effects minimization

- Attainable Utility Preservation (AUP)

- AUP and alignment

- Power-seeking

- Power-seeking and alignment

- Future work and about Alex

The transcript: axrp.net/episode/2021/09/25/episode-11-attainable-utility-power-alex-turner.html

Alex on the AI Alignment Forum: alignmentforum.org/users/turntrout

Alex's Google Scholar page: scholar.google.com/citations?user=thAHiVcAAAAJ&hl=en&oi=ao

Conservative Agency via Attainable Utility Preservation: arxiv.org/abs/1902.09725

Optimal Policies Tend to Seek Power: arxiv.org/abs/1912.01683

Other works discussed:

- Avoiding Side Effects by Considering Future Tasks: arxiv.org/abs/2010.07877

- The "Reframing Impact" Sequence: alignmentforum.org/s/7CdoznhJaLEKHwvJW

- The "Risks from Learned Optimization" Sequence: alignmentforum.org/s/7CdoznhJaLEKHwvJW

- Concrete Approval-Directed Agents: ai-alignment.com/concrete-approval-directed-agents-89e247df7f1b

- Seeking Power is Convergently Instrumental in a Broad Class of Environments: alignmentforum.org/s/fSMbebQyR4wheRrvk/p/hzeLSQ9nwDkPc4KNt

- Formalizing Convergent Instrumental Goals: intelligence.org/files/FormalizingConvergentGoals.pdf

- The More Power at Stake, the Stronger Instumental Convergence Gets for Optimal Policies: alignmentforum.org/posts/Yc5QSSZCQ9qdyxZF6/the-more-power-at-stake-the-stronger-instrumental

- Problem Relaxation as a Tactic: alignmentforum.org/posts/JcpwEKbmNHdwhpq5n/problem-relaxation-as-a-tactic

- How I do Research: lesswrong.com/posts/e3Db4w52hz3NSyYqt/how-i-do-research

- Math that Clicks: Look for Two-way Correspondences: lesswrong.com/posts/Lotih2o2pkR2aeusW/math-that-clicks-look-for-two-way-correspondences

- Testing the Natural Abstraction Hypothesis: alignmentforum.org/posts/cy3BhHrGinZCp3LXE/testing-the-natural-abstraction-hypothesis-project-intro

Remember Everything You Learn from Podcasts

Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.