AXRP - the AI X-risk Research Podcast

4 - Risks from Learned Optimization with Evan Hubinger

Feb 17, 2021

02:13:32

Creator website

Highlights

AI Chapters

Episode notes

How to Solve a Problem Before the Observation

01:30

Do You Know How to Train Look Up Tables?

03:53

Is There a Difference Between a Model's Mas Objective and a Behavior Objective?

03:27

Introduction

2min

What's the Difference Between Mesa and Meta in Machine Learning?

2min

Is Optimization a Mechanistic Process?

3min

Neural Networks

6min

Will It Generalize Properly to This New Datapoint?

4min

Generalization Bounds Aren't Enough, You Know?

2min

The Deceptive Linear Training Set Up No Longer Works

4min

Admosperal Training

6min

What Is a Messimess Optimiser?

3min

10.

Openin Five Agents - Is There a Coherent Objective?

2min

11.

Is There a Behaviour Objective?

3min

12.

Is There a Difference Between a Model's Mas Objective and a Behavior Objective?

3min

13.

Is There an Open Research Problem in Mecha Optimisation?

3min

14.

Do You Expect Optimization to Happen at Training Time?

5min

15.

How Simple Are Optimisations in Machine Learning?

6min

16.

Is It a Speed Prior?

2min

17.

Is There a Difference Between Time and Space?

3min

18.

Masoptimizer

2min

19.

Do You Know How to Train Look Up Tables?

4min

20.

Is There a Way to Get a Masoprex?

5min

21.

Do You Want to Avoid Mesooptimizers?

4min

22.

Machine Learning and Machine Learning in the Machine Learning Community

2min

23.

Is There a Way to Decompose a Module?

4min

24.

Machine Learning and Inner Alignment

5min

25.

Sub Optomal at Being Deceptive?

2min

26.

Proxy Linement Failures

4min

27.

Is Sttubing My Toe Bad?

2min

28.

What You Really Care About Is, Like, the Alil Frequency of A.

2min

29.

Do You Want to Train a System for a Diverse Set of Tasks?

3min

30.

Is It a Trade Worth Making?

6min

31.

Is There an Inner Alignment Problem?

2min

32.

Do You Get Deceptive Alignment?

4min

33.

Training on a Huge Corpus of Data

2min

34.

How Many Martin Lutherthers Are There?

2min

35.

How Do You Get a Situation Where Your System Cares About Multile Episodes?

5min

36.

Nonmaapia Problems in on Line Learning

3min

37.

How Confident Are You That the Arguments in the Paper Are Correct?

2min

38.

The Most Dangerous Outcomes From Ai

3min

39.

What Do We Do About This?

2min

40.

How to Solve a Problem Before the Observation

2min

41.

How to Follow Your Work?

2min

In machine learning, typically optimization is done to produce a model that performs well according to some metric. Today's episode features Evan Hubinger talking about what happens when the learned model itself is doing optimization in order to perform well, how the goals of the learned model could differ from the goals we used to select the learned model, and what would happen if they did differ.

Link to the paper - Risks from Learned Optimization in Advanced Machine Learning Systems: arxiv.org/abs/1906.01820

Link to the transcript: axrp.net/episode/2021/02/17/episode-4-risks-from-learned-optimization-evan-hubinger.html

Evan Hubinger's Alignment Forum profile: alignmentforum.org/users/evhub

Remember Everything You Learn from Podcasts

Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.