Generally Intelligent

Episode 13: Jonathan Frankle, MIT, on the lottery ticket hypothesis and the science of deep learning

Sep 10, 2021

01:20:33

Creator website

Highlights

AI Chapters

Episode notes

The Unappealing Style of Work: swinging for the fences

01:10

Pruning Networks of Initialization: A Review

01:16

Open Review: How to be a good reviewer

01:48

Object Oriented Programming

02:19

Introduction

2min

Machine Learning - What's the Story of Your Research Interests?

4min

Did You Have Research Interests?

3min

Can You Be a Great Researcher Without Being the World's Greatest Mathematician?

3min

Why You Can't Train a Prude Network?

4min

Is There a Way to Start Small?

3min

How Do You Preserve Your Performance With Network Pruning?

4min

Is Early in Training Really Early?

4min

Prue Networks of Initialization - Is This a Good Technique?

3min

10.

Deep Learning - What Are Some of the Most Important Open Questions?

3min

11.

Machine Learning

2min

12.

Those Roses After You Found an Idea That Worked

5min

13.

Deep Learning

2min

14.

I'm Not Trying to Be the First Person to Have the Next Advance

2min

15.

How Do You Find More Papers Like This?

3min

16.

I Think We Need More Than That in the Field of Science

4min

17.

Manifesto for Papers

2min

18.

The Original Lottery Ticket Paper

2min

19.

Grat Lt, Bu Wa O Yotot?

4min

20.

Should We Use Facial Recognition?

2min

21.

Why Do We Write Assembles?

3min

22.

I'm a Scientist, and I Build on What I See in Front of Me

2min

23.

Object Oriented Programming

2min

24.

The Hardest Thing for a Junior Researcher

2min

25.

Publication Doesn't Have to Be Significant

2min

26.

The Story Behind the Bachnorm Paper

2min

27.

Why Is Sparsity Everywhere?

2min

28.

I'm Not a Lawyer, but I Want to Know.

2min

29.

Is There Any Scale Set for Machine Learning?

2min

30.

How to Train 700 Networks on Image Net?

4min

Jonathan Frankle (Google Scholar) (Website) is finishing his PhD at MIT, advised by Michael Carbin. His main research interest is using experimental methods to understand the behavior of neural networks. His current work focuses on finding sparse, trainable neural networks.

**Highlights from our conversation:**

🕸 "Why is sparsity everywhere? This isn't an accident."

🤖 "If I gave you 500 GPUs, could you actually keep those GPUs busy?"

📊 "In general, I think we have a crisis of science in ML."

Remember Everything You Learn from Podcasts

Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.