The Data Exchange with Ben Lorica

Evaluating Language Models

Jan 19, 2023

45:36

AI Chapters

Episode notes

Introduction

2min

The Holistic Evaluation of Language Models

2min

How to Test a Language Model for General Purpose Tasks

3min

The Seven Metrics of Detoxification Toxicity

2min

How to Score Language Models in a Benchmark

2min

How to Use Helm to Narrow Down Your Use Cases

2min

The Future of Helm: A Community Initiative

2min

The Importance of Scaling Language Models

2min

The Complexity of Copyright

3min

10.

The Challenges of Evaluating Language Models

1min

11.

The Gap Between Open AI and Private Models

2min

12.

How to Measure Efficiency in the API

1min

13.

The Future of Language Models

3min

14.

The Promise of Language Models

2min

15.

Fine Tuning General Purpose Language Models

2min

16.

The Paradigm Shift in Foundation Models

2min

17.

The Future of Foundation Models

2min

18.

The Importance of Data

2min

19.

The Importance of Externalizing Knowledge

3min

20.

Helm: A Center for Foundation Models

6min

Percy Liang is Associate Professor of Computer Science and Statistics, and Director of the new Center for Research on Foundation Models at Stanford University. We discussed a new suit of tools (HELM) designed to help users and researchers understand language models in their totality. We also discuss recent trends in AI including the rise of Generative AI and Foundation Models.

Download a copy of our FREE 2023 Trends in Data and AI Report: https://gradientflow.com/2023trendsreport/

Subscribe: Apple • Spotify • Stitcher • Google • AntennaPod • Podcast Addict • Amazon • RSS.

Detailed show notes can be found on The Data Exchange web site.

Get the Snipd
podcast app

Unlock the knowledge in podcasts with the podcast player of the future.

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode

Save any
moment

Hear something you like? Tap your headphones to save it with AI-generated key takeaways

Save any
moment

Hear something you like? Tap your headphones to save it with AI-generated key takeaways

Share
& Export

Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more

Share
& Export

Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode

The Data Exchange with Ben Lorica

Evaluating Language Models

Get the Snipdpodcast app

AI-poweredpodcast player

AI-poweredpodcast player

Discoverhighlights

Discoverhighlights

Save anymoment

Save anymoment

Share& Export

Share& Export

AI-poweredpodcast player

AI-poweredpodcast player

Discoverhighlights

Discoverhighlights

Get the Snipd
podcast app

AI-powered
podcast player

AI-powered
podcast player

Discover
highlights

Discover
highlights

Save any
moment

Save any
moment

Share
& Export

Share
& Export

AI-powered
podcast player

AI-powered
podcast player

Discover
highlights

Discover
highlights