Hear This Idea

#63 – Ben Garfinkel on AI Governance

May 13, 2023

02:58:08

Episode guests

Ben Garfinkel

Highlights

AI Chapters

Episode notes

The Implications of AI on Human Well-Being

01:28

The Role of Technology in Conflict

04:05

The Probability of a Global Catastrophe

02:54

The Probability of Different Harms Caused by Unsafe AI Systems

02:33

Introduction

2min

The Impacts of Artificial Intelligence on the World

2min

The Potential Impacts of AI on Human Well-Being

5min

The Role of Technology in Conflict

4min

The Effects of the Industrial Revolution on the Nature of Productivity

5min

The Importance of International Coordination

2min

The Risks of Out of Control AI Systems

5min

The Story of Contingency in AI

2min

The Contingency Story of the Cold War

3min

10.

The Noise of Nuclear War in the 1950s

3min

11.

The Tradeoff Between Commercial Pressure and Safety

6min

12.

The Probability of a Global Catastrophe

3min

13.

The Differences Between International Competitive Dynamics and Domestic Relations

5min

14.

The Problem With AI Governance

2min

15.

The Future of AI Governance

4min

16.

The Differences Between the Harms and the Benefits of AI

2min

17.

The Transition to Advanced AI

3min

18.

The Differences Between People Working in Governments Motivated by Long-Term Concerns

2min

19.

The Importance of Paying Attention to Long-Term Impacts of AI

4min

20.

The Perception of Unsafe AI Systems

3min

21.

The Differentiation Effects of AI Views

3min

22.

How Financial Markets Respond to AGI Catastrophes

2min

23.

The Selection Effect of Financial Markets

3min

24.

The Range of Views on Financial Markets

2min

25.

How Much Weight Should You Give to Observations About Present Day Air Systems?

3min

26.

The Importance of Skepticism in AI Arguments

3min

27.

The Probability of Different Harms Caused by Unsafe AI Systems

3min

28.

The Multiple Equilibriums of Frontier Systems

3min

29.

The Mechanistic Story of Risk

5min

30.

The Winner's Curse of Randomness

2min

31.

The Propensity for Catastrophes in AI Systems

2min

32.

The Argument for a Single AI System

5min

33.

The Importance of a Skeptical Prior

3min

34.

The Discontinuity of AI Development Processes

2min

35.

The Orthogonality Thesis and the Argument for Instrumental Convergence

2min

36.

The Abstract Instrumental Convergence Argument for Machine Learning

4min

37.

The Limits of Artificial Intelligence Progress

3min

38.

Doom Is High in Hindsight

3min

39.

The Future of AI

3min

40.

The Pros and Cons of Negative Feedback From AI Systems

3min

41.

How to Handle a Situation Where You Have Deceptively Misaligned Systems

3min

42.

Human Evolution History and the Septic Misalignment

5min

43.

The Bi-Nology of AI Systems

4min

44.

The Importance of Perception in AI Systems

3min

45.

Evolutionary History and the Preferences People Have Today

4min

46.

The Evolution of Large Language Models

3min

47.

The Strange Position of the Brother World in the ML Space

3min

48.

The Consequences of Estimating the Chance of Human Extinction

3min

49.

The Future of AI Governance

6min

50.

The Role of AGI Systems in Legislation

2min

51.

The Importance of a Basket of Skills

5min

52.

How to Be a Successful Biosecurity Policy Expert

5min

53.

The Frontier Isn't Moved Very Far Forward

2min

54.

The Limits of Risks From Advanced AI Systems

2min

55.

Gavey I: A Venture Fellow's Perspective on the Future of Training

2min

56.

The Future of AI Governance

4min

57.

How to Find You in Gaveye Online

2min

58.

Ben Garfinkel on AI Governance

2min

Ben Garfinkel is a Research Fellow at the University of Oxford and Acting Director of the Centre for the Governance of AI.

In this episode we talk about:

An overview of AI governance space, and disentangling concrete research questions that Ben would like to see more work on
Seeing how existing arguments for the risks from transformative AI have held up and Ben’s personal motivations for working on global risks from AI
GovAI’s own work and opportunities for listeners to get involved

Further reading and a transcript is available on our website: hearthisidea.com/episodes/garfinkel

If you have any feedback, you can get a free book for filling out our new feedback form. You can also get in touch through our website or on Twitter. Consider leaving us a review wherever you're listening to this — it's the best free way to support the show. Thanks for listening!

Remember Everything You Learn from Podcasts

Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.