5min chapter

Deep Papers cover image

Toolformer: Training LLMs To Use Tools

Deep Papers

CHAPTER

Using Pre-Trained Language Models to Generate API Calls

Teemo: One of the problems with this is getting a huge annotated dataset. And part of the problem is because the scale of these text datasets is so large, like common crawl, you can't just pay human labelers to do it. We wanted to avoid having huge costs from humans having to annotate data. So we use a language model to sample a massive amount of different API calls. Then we give it three examples of an input and the corresponding output would be the same text with API calls inserted. And then we ask the model to do the same for a fourth text.

00:00

Get the Snipd
podcast app

Unlock the knowledge in podcasts with the podcast player of the future.
App store bannerPlay store banner

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode

Save any
moment

Hear something you like? Tap your headphones to save it with AI-generated key takeaways

Share
& Export

Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode