Deep Papers

ChatGPT and InstructGPT: Aligning Language Models to Human Intention

Jan 18, 2023

47:39

Episode guests

Ryan Lowe

Long Ouyang

Highlights

AI Chapters

Episode notes

Is There a Future for Machine Learning?

02:59

How to Train a Reward Model for a Good Job

02:34

The Nature of Prompt Engineering Will Change Over Time

02:00

Introduction

2min

ML Observability and Alignment - Part 1 of 3

2min

The Main Problems With Large Language Models

2min

Is the Term Alignment Created by OpenAI?

2min

Instruct GPT Paper - What Is It?

2min

How Do You Train a Reward Model?

2min

How to Train a Reward Model for a Good Job

3min

Do You See Other Major Applications Just Skipping the First Step?

2min

How Did You Come Up With the Idea of Instructing GPT?

2min

10.

The Nature of Prompt Engineering Will Change Over Time

2min

11.

Are There Other Major Benefits of LHF?

4min

12.

OpenAI

2min

13.

Is There a Jury on Large Language Models?

2min

14.

The Next Generation of Language Models Are Going to Be Really Really Powerful

2min

15.

Is There a Future for Machine Learning?

3min

16.

How Much Is the Reward Model the Most Important?

2min

17.

Is LHF the Best Way to Fine Tune Language Models?

2min

18.

Is There a Way to Evaluate a Powerful Model?

2min

19.

The Long Term Alignment Research

2min

20.

Is There Anything You Can Point to That's Not Really Great?

2min

21.

Is That Part of the Actual Training or Is That Like the Fine Tuning?

3min

22.

I'd Like to Give Chat and Instruct GPT a Try

2min

Deep Papers is a podcast series featuring deep dives on today’s seminal AI papers and research. Hosted by AI Pub creator Brian Burns and Arize AI founders Jason Lopatecki and Aparna Dhinakaran, each episode profiles the people and techniques behind cutting-edge breakthroughs in machine learning.

In this first episode, we’re joined by Long Ouyang and Ryan Lowe, research scientists at OpenAI and creators of InstructGPT. InstructGPT was one of the first major applications of Reinforcement Learning with Human Feedback to train large language models, and is the precursor to the now-famous ChatGPT. Listen to learn about the major ideas behind InstructGPT and the future of aligning language models to human intention.

Read OpenAI's InstructGPT paper here: https://openai.com/blog/instruction-following/

Learn more about AI observability and evaluation, join the Arize AI Slack community or get the latest on LinkedIn and X.

Remember Everything You Learn from Podcasts

Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.