Machine Learning Street Talk (MLST) cover image

Sara Hooker - Why US AI Act Compute Thresholds Are Misguided

Machine Learning Street Talk (MLST)

NOTE

Optimizing Model Capacity in Multilingual Settings

In multilingual settings, careful consideration must be given to model capacity optimization due to the challenges of handling various underrepresented languages. A 13 billion parameter model used in 101 was challenging to adapt due to a lack of pre-training data. This necessitated meticulous data processing, data cleaning, and working with synthetic data to manage optimization time. Options include switching to a three billion parameter model with retraining or adopting a more resourceful approach to optimization and data creation. Challenges in multilingual settings are compounded, requiring attention to details and a thoughtful strategy for capacity utilization.

00:00
Transcript
Play full episode

Remember Everything You Learn from Podcasts

Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.
App store bannerPlay store banner