The Data Exchange with Ben Lorica

Efficient Methods for Natural Language Processing

Dec 1, 2022

45:40

Highlights

AI Chapters

Episode notes

Pre-Trained Models - Can I Apply This to Any Kind of Domain?

02:17

NLP, Data Set Creators and Data Set Engineers

02:08

Is There a Tool That Can Surface Biases?

02:19

Introduction

5min

How to Scale Out Efficiency in Deep Learning?

2min

Using 100% of the Data, You're Not Losing That Much

2min

Pre-Trained Models - Can I Apply This to Any Kind of Domain?

2min

The High Level Takeaway for Efficiency Methods

3min

Is There a Trade Off Between Expert and Mixed Approach?

2min

How Do You Reduce the Efficiency of Your Model?

2min

Pre-Training and Fine Tuning in NLP

2min

Using Labeled Examples to Fine Tune Your Model

3min

10.

The Challenge of Fine Tuning GPUs

2min

11.

Evaluating Models Like GPD-3

2min

12.

NLP Models - What Are the First Two to Three Things You Would Try?

2min

13.

NLP

1min

14.

NLP, Data Set Creators and Data Set Engineers

2min

15.

80% of Energy Costs Are on the Inference Side, Right?

2min

16.

How Much Interest and Awareness Is There in Green AI?

2min

17.

How Much It Costs to Train Large Neural Networks?

2min

18.

Detoxification of Language Models

2min

19.

Is There a Bias in Data Sets?

2min

20.

Is There a Tool That Can Surface Biases?

2min

21.

The Data Exchange Podcast Is a Property of Gradient Flow

2min

Roy Schwartz is Professor of Natural Language Processing at The Hebrew University of Jerusalem. We discussed a recent survey paper that Roy co-wrote that presented a broad overview of existing methods to improve NLP efficiency through the lens of traditional NLP pipelines.

Subscribe to the Gradient Flow Newsletter: https://gradientflow.substack.com/

Subscribe: Apple • Android • Spotify • Stitcher • Google • AntennaPod • RSS.

Detailed show notes can be found on The Data Exchange web site.