The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence) cover image

Multilingual LLMs and the Values Divide in AI with Sara Hooker - #651

The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)

CHAPTER

Multilingual Model Development Insights

This chapter compares the Big Science Project and the Bloom initiative in their approaches to pre-training language models, focusing on enhancing multilingual representation. It addresses the technical challenges of optimizing data quality and representation in multilingual contexts while discussing the benefits of joint training for low-resource languages. The chapter concludes by highlighting the scaling plans for their MT5 architecture and the advantages of instruction fine-tuning over conventional methods.

00:00
Transcript
Play full episode

Remember Everything You Learn from Podcasts

Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.
App store bannerPlay store banner