6min chapter

No Priors: Artificial Intelligence | Technology | Startups cover image

The AI Will See You Now: Exploring Biomedical AI and Google’s Med-PaLM2 With Karan Singhal

No Priors: Artificial Intelligence | Technology | Startups

CHAPTER

The Importance of Optimizing Pre-Training for Multilingual Data

This is work that started with UL2, a paper that was released also last year. The total amount of data being used for these models was relatively low compared to the number of parameters. This especially focused on multilingual data and more code data in a bunch of different coding languages as well. We're just zooming out a little bit in terms of when you might apply some of these different techniques to align a model to a specific domain. And I think it really comes down to the data that's available, both in quantity and relevance to a particular topic.

00:00

Get the Snipd
podcast app

Unlock the knowledge in podcasts with the podcast player of the future.
App store bannerPlay store banner

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode

Save any
moment

Hear something you like? Tap your headphones to save it with AI-generated key takeaways

Share
& Export

Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode