Is There a Constraint in Ai Systems?

i think that going about it via thinking in like, precise technical terms about the sort of limit of intelligence seems a little bit off. It feels like my thinking here is about handing over to ai systems that are alined enough that they can do the heavy lifting in te future. And i don't think it's like necessarily a bad strategy to focus on just, like, solving the whole problem in one go.

Transcript

chevron_right

Play full episode

chevron_right

Transcript

Episode notes

How should we think about artificial general intelligence (AGI), and the risks it might pose? What constraints exist on technical solutions to the problem of aligning superhuman AI systems with human intentions? In this episode, I talk to Richard Ngo about his report analyzing AGI safety from first principles, and recent conversations he had with Eliezer Yudkowsky about the difficulty of AI alignment.

Topics we discuss, and timestamps:

- 00:00:40 - The nature of intelligence and AGI

- 00:01:18 - The nature of intelligence

- 00:06:09 - AGI: what and how

- 00:13:30 - Single vs collective AI minds

- 00:18:57 - AGI in practice

- 00:18:57 - Impact

- 00:20:49 - Timing

- 00:25:38 - Creation

- 00:28:45 - Risks and benefits

- 00:35:54 - Making AGI safe

- 00:35:54 - Robustness of the agency abstraction