The Data Exchange with Ben Lorica

Evaluating Language Models

Jan 19, 2023
Ask episode
Chapters
Transcript
Episode notes
1
Introduction
00:00 • 2min
2
The Holistic Evaluation of Language Models
01:58 • 2min
3
How to Test a Language Model for General Purpose Tasks
04:23 • 3min
4
The Seven Metrics of Detoxification Toxicity
07:07 • 2min
5
How to Score Language Models in a Benchmark
08:54 • 2min
6
How to Use Helm to Narrow Down Your Use Cases
11:06 • 2min
7
The Future of Helm: A Community Initiative
13:01 • 2min
8
The Importance of Scaling Language Models
15:08 • 2min
9
The Complexity of Copyright
17:18 • 3min
10
The Challenges of Evaluating Language Models
19:54 • 1min
11
The Gap Between Open AI and Private Models
21:23 • 2min
12
How to Measure Efficiency in the API
22:57 • 1min
13
The Future of Language Models
24:26 • 3min
14
The Promise of Language Models
27:39 • 2min
15
Fine Tuning General Purpose Language Models
29:09 • 2min
16
The Paradigm Shift in Foundation Models
30:52 • 2min
17
The Future of Foundation Models
32:57 • 2min
18
The Importance of Data
35:23 • 2min
19
The Importance of Externalizing Knowledge
37:15 • 3min
20
Helm: A Center for Foundation Models
39:53 • 6min