The Data Exchange with Ben Lorica

Exhaustion of High-Quality Data Could Slow Down AI Progress in Coming Decades

Mar 16, 2023

33:09

AI Chapters

Episode notes

Introduction

2min

The Scaling Loss of a Model

2min

The Key Findings for NLP

3min

The Future of Computer Vision

1min

The Reaction to Your Paper From Machine Learning Researchers

2min

Scaling Loss and the Importance of Data

3min

The State of Scaling Loss in Machine Learning

3min

Do You Track Friends in Memory?

2min

The Trends in Model Size

2min

10.

The Role of Synthetic Data in Artificial Intelligence

2min

11.

The Future of Retrieval Mechanisms

2min

12.

The Unpredictabilities of Transfer Learning

2min

13.

The Rise of Generative AI

2min

14.

The Future of Open Source Data

4min

Pablo Villalobos is a Staff Researcher at Epoch, and lead author of the recent paper “Will we run out of data? An analysis of the limits of scaling datasets in Machine Learning”. We discuss the key findings in this paper, as well as a related study Pablo conducted on scaling laws.

Subscribe to the Gradient Flow Newsletter: https://gradientflow.substack.com/

Subscribe: Apple • Spotify • Stitcher • Google • AntennaPod • Podcast Addict • Amazon • RSS.

Detailed show notes can be found on The Data Exchange web site.

Get the Snipd
podcast app

Unlock the knowledge in podcasts with the podcast player of the future.

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode

Save any
moment

Hear something you like? Tap your headphones to save it with AI-generated key takeaways

Share
& Export

Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode

The Data Exchange with Ben Lorica

Exhaustion of High-Quality Data Could Slow Down AI Progress in Coming Decades

Get the Snipdpodcast app

AI-poweredpodcast player

Discoverhighlights

Save anymoment

Share& Export

AI-poweredpodcast player

Discoverhighlights

Get the Snipd
podcast app

AI-powered
podcast player

Discover
highlights

Save any
moment

Share
& Export

AI-powered
podcast player

Discover
highlights