Last Week in AI

OpenAI's "Scaling Laws for Autoregressive Generative Modeling"

Nov 8, 2020

33:15

Creator website

AI Chapters

Episode notes

Introduction

4min

The Parallel Relationship Between Model Loss and Cross Entropy

2min

The Relationship Between Reducible and Irreducible Loss

2min

The Importance of Reducible Loss

2min

The Optimal Model Size for a Given Compute Budget Is a Power Law

3min

The Power of Reducible Loss in Image Net Classification

4min

The Easiest Way to Optimize Larger Models

4min

The Inconsistency in Scaling Laws for Data Set Size

4min

The Power of Neural Networks in Machine Learning

3min

10.

The Evolution of the Decoder Only Transformer in Language Modeling

3min

11.

Scaling Loss for Autoregressive Generative Modeling

2min

An interview with Tom Henighan, a member of the technical staff at OpenAI working on the safety team, about the recent paper “Scaling Laws for Autoregressive Generative Modeling” that he co-authored with many others at OpenAI.

Subscribe: RSS | iTunes | Spotify | YouTube

Check out coverage of similar topics at www.skynettoday.com

Theme: Deliberate Thought Kevin MacLeod (incompetech.com)

Get the Snipd
podcast app

Unlock the knowledge in podcasts with the podcast player of the future.

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode

Save any
moment

Hear something you like? Tap your headphones to save it with AI-generated key takeaways

Share
& Export

Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode

Last Week in AI

OpenAI's "Scaling Laws for Autoregressive Generative Modeling"

Get the Snipdpodcast app

AI-poweredpodcast player

Discoverhighlights

Save anymoment

Share& Export

AI-poweredpodcast player

Discoverhighlights

Get the Snipd
podcast app

AI-powered
podcast player

Discover
highlights

Save any
moment

Share
& Export

AI-powered
podcast player

Discover
highlights