TalkRL: The Reinforcement Learning Podcast

John Schulman

Oct 18, 2022

44:21

Creator website

Episode guests

John Schulman

Highlights

AI Chapters

Episode notes

What Language Models Can Learn From the Internet?

03:07

Using an on Policy Algorithm With a Reward Model

02:19

WebGPT - Is There a Line Between Knowledge and Understanding?

01:58

Introduction

2min

Using Human Feedback to Define a Reward Function

2min

Web Assisted Question Answering With Human Feedback

4min

WebGBT - What Inspired the Idea?

3min

RL From Human Feedback?

2min

Do You Think the RL Task Is Going to Work?

4min

WebGPT - Is There a Line Between Knowledge and Understanding?

2min

Instruct GPT - A Language Model That's Fine-Tuned to Follow Instructions

4min

What Language Models Can Learn From the Internet?

3min

10.

Generalization and Reward Models in Machine Learning

2min

11.

Can Language Models Tell Us When They Don't Know What They Know?

2min

12.

How Do You See AI Alignment?

2min

13.

Uh, Is the Standard RL Framing Problematic?

2min

14.

The Relationship Between AGI and RL

2min

15.

Is There a Time to Rethink PPO?

2min

16.

Patents in Machine Learning Algorithms?

2min

17.

What Are the Biggest Highlights in RL?

2min

18.

Using an on Policy Algorithm With a Reward Model

2min

19.

The Future of RL Algorithms

2min

John Schulman is a cofounder of OpenAI, and currently a researcher and engineer at OpenAI.

Featured References

WebGPT: Browser-assisted question-answering with human feedback
Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, Xu Jiang, Karl Cobbe, Tyna Eloundou, Gretchen Krueger, Kevin Button, Matthew Knight, Benjamin Chess, John Schulman

Training language models to follow instructions with human feedback
Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, Ryan Lowe

Additional References