2min chapter

The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence) cover image

Towards Improved Transfer Learning with Hugo Larochelle - #631

The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)

CHAPTER

How to Pre-Trained Large Language Models

The thought was that you'd be pulling in these snippets that were taken out of context and not complete like the kind of data the LLM was trained on. So again like to me a lot of these projects are also about trying to uncover interesting properties of these various ways we can pre-trained large models from lots of data. And here it was kind of interesting to see that it had some amount of robustness, more than robustness it's actually can if you put in better information it would actually leverage it.

00:00

Get the Snipd
podcast app

Unlock the knowledge in podcasts with the podcast player of the future.
App store bannerPlay store banner

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode

Save any
moment

Hear something you like? Tap your headphones to save it with AI-generated key takeaways

Share
& Export

Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode