Clearer Thinking with Spencer Greenberg

What, if anything, do AIs understand? (with ChatGPT Co-Creator Ilya Sutskever)

Oct 26, 2022

01:15:30

Creator website

Highlights

AI Chapters

Episode notes

Comparing GPT-3 and the Human Brain: Similarities and Differences

01:08

Two ways to define intelligence and the history of AI advancements

01:44

The Importance of Understanding Prediction

01:06

The Training Paradox: More Data vs. More Parameters in Neural Networks

01:11

Introduction

2min

The Importance of General Purpose Tools in Deep Learning

5min

The Connection Between Predicting and Understanding

3min

How to Overcome the Barrier to GP3

3min

How Deep Neural Networks Can Help You Train a System

5min

The Power of Transformers in Machine Learning

2min

The Belief in Large Networks

2min

The Sociology of Machine Learning

2min

The Importance of Simple Methods in AI

3min

10.

The Future of Academic Research in AI

2min

11.

The Importance of Intuition in Model Learning

3min

12.

The Importance of Generalization in Neural Networks

3min

13.

The Perplexing Problem With Bayesian Inference

2min

14.

The Future of AI Systems

3min

15.

The Difference Between GPT-3 and the Human Brain

2min

16.

The Differences Between GPT3 and Humans

2min

17.

The Gap Between Human Learning and GPT Learning

2min

18.

The Importance of Symbolic Reasoning in Computers

3min

19.

The Universal Approximation of Neural Networks

2min

20.

The Importance of Answering Open-Ended Questions

2min

21.

AI and the Dangers It Possesses

2min

22.

The Potential Dangers of AI

3min

23.

OpenAI: A Collective Action Problem

2min

24.

The Future of Artificial Intelligence

3min

25.

The Dangers of Concentrating Power in AI

4min

26.

The Future of AI

2min

27.

The Dangers of a Truly Super Intelligent AI

2min

28.

The Evolution of Open Source in AI

2min

29.

The Importance of Causation in AI

3min

Read the full transcript here.

Can machines actually be intelligent? What sorts of tasks are narrower or broader than we usually believe? GPT-3 was trained to do a "single" task: predicting the next word in a body of text; so why does it seem to understand so many things? What's the connection between prediction and comprehension? What breakthroughs happened in the last few years that made GPT-3 possible? Will academia be able to stay on the cutting edge of AI research? And if not, then what will its new role be? How can an AI memorize actual training data but also generalize well? Are there any conceptual reasons why we couldn't make AIs increasingly powerful by just scaling up data and computing power indefinitely? What are the broad categories of dangers posed by AIs?

Ilya Sutskever is Co-founder and Chief Scientist of OpenAI, which aims to build artificial general intelligence that benefits all of humanity. He leads research at OpenAI and is one of the architects behind the GPT models. Prior to OpenAI, Ilya was co-inventor of AlexNet and Sequence to Sequence Learning. He earned his Ph.D. in Computer Science from the University of Toronto. Follow him on Twitter at @ilyasut.

Staff