AXRP - the AI X-risk Research Podcast

10 - AI's Future and Impacts with Katja Grace

Jul 23, 2021

Ask episode

Chapters

Transcript

Episode notes

Is There a Fast Explosion?

How Do You Decide Which Sub Questions to Answer?

What Do You Think About Non Epistemic Features?

Is There a Difference Between a I and an I?

Is There a Difference Between a I and Other Things?

When Willi Overcome Human Performance?

When Human Level a I Might Materialize?

What's the Difference in Probability?

Is There a Difference in Framing?

Is High Level Machine Intelligence Really Imperative?

How Much Progress Have You Seen in the Field in How Many Years?

Forecasting Machine Intelligence

What Is the Most Interesting Finding From Your Survey?

How Long Does It Take to Cross the Human Skill Range?

Is There a Difference Between Humans and Other Animals?

Are Chickens Really Good at Checkersbord?

The Range in Learning Languages Isn't That Big

How Long Will It Take to Measure Time?

How Common Are Discontinuities in Technology?

Is There a Discontinuity?

How Many Discontinuities Are There in Ai?

Is There a Discontinuity?

Is Machine Learning a Good Idea for Machine Learning?

Is There a Human Skill Range, or a Simple Algorithm?

Is There a Difference Between Human Ability and Human Ability?

Is There a Core Intelligence to Doing Well at Schmess?

01:01:37 • 2min

Is There a Risk in a I?

01:03:17 • 3min

Is There a Better Model for Incoherent Preferences?

01:06:41 • 2min

How Do You Become More Coherent?

01:09:02 • 2min

Are You Imagining the Future Choice?

01:10:36 • 2min

Would You Like to Have a Superhuman?

01:12:32 • 2min

Is There a Level of Progress in Humans?

01:14:38 • 2min

Optimizing It Doesn't Take Over the World

01:16:59 • 2min

Is It Like, Make Me a Breter, Please?

01:18:37 • 2min

Agents Are Like, Really Good for Achieving Goals

01:21:04 • 2min

How to Write a Fore Loop Into a Gamer's Environment

01:22:44 • 2min

Is There a Difference Between Agents and Humans?

01:24:37 • 3min

Is There a Range of Goals in Programming?

01:27:26 • 2min

I've Changed My Mind About Ai

01:29:51 • 3min

Is There a Future That We Hate?

01:33:05 • 2min

The Most Competent People Aren't Doing That Well, Right?

01:35:09 • 2min

Can You Trick All Animals?

01:37:12 • 2min

The Relationship Between Agents and Agents

01:38:46 • 2min

How Much of the Future Do You Get?

01:40:21 • 2min

Is There a Lot of Control of the Future?

01:42:01 • 2min

Ai Systems Respect Property Rights

01:43:44 • 2min

Is It a Thing Like Human Extinction?

01:45:39 • 1min

Is It Human Extinction?

01:47:01 • 3min

Is This a Risk?

01:49:43 • 2min

How Long Does It Take to Solve Various at Conjectures?

01:51:41 • 2min

Is There a Difference in the Kind of Clusters of Related Stuff in Apex?

01:53:18 • 3min

I Think It's the Point of a PhD Theses, Theses

01:56:08 • 2min

The Difference Between Ai and Academia

01:57:50 • 2min

A Impacts Dotr Web Site, Is It a Good Idea?

02:00:13 • 3min

When going about trying to ensure that AI does not cause an existential catastrophe, it's likely important to understand how AI will develop in the future, and why exactly it might or might not cause such a catastrophe. In this episode, I interview Katja Grace, researcher at AI Impacts, who's done work surveying AI researchers about when they expect superhuman AI to be reached, collecting data about how rapidly AI tends to progress, and thinking about the weak points in arguments that AI could be catastrophic for humanity.

Topics we discuss:

- 00:00:34 - AI Impacts and its research

- 00:08:59 - How to forecast the future of AI

- 00:13:33 - Results of surveying AI researchers

- 00:30:41 - Work related to forecasting AI takeoff speeds

- 00:31:11 - How long it takes AI to cross the human skill range

- 00:42:47 - How often technologies have discontinuous progress

- 00:50:06 - Arguments for and against fast takeoff of AI

- 01:04:00 - Coherence arguments

- 01:12:15 - Arguments that AI might cause existential catastrophe, and counter-arguments

- 01:13:58 - The size of the super-human range of intelligence

- 01:17:22 - The dangers of agentic AI

- 01:25:45 - The difficulty of human-compatible goals

- 01:33:54 - The possibility of AI destroying everything

- 01:49:42 - The future of AI Impacts

- 01:52:17 - AI Impacts vs academia

- 02:00:25 - What AI x-risk researchers do wrong

- 02:01:43 - How to follow Katja's and AI Impacts' work

The transcript: axrp.net/episode/2021/07/23/episode-10-ais-future-and-dangers-katja-grace.html

"When Will AI Exceed Human Performance? Evidence from AI Experts": arxiv.org/abs/1705.08807

AI Impacts page of more complete survey results: aiimpacts.org/2016-expert-survey-on-progress-in-ai

Likelihood of discontinuous progress around the development of AGI: aiimpacts.org/likelihood-of-discontinuous-progress-around-the-development-of-agi

Discontinuous progress investigation: aiimpacts.org/discontinuous-progress-investigation

The range of human intelligence: aiimpacts.org/is-the-range-of-human-intelligence-small