AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
1min snip
#229 Inside Meta's Biggest and Best Open-Source AI Model Yet with Thomas Scialom, Co-Creator of Llama3
DataFramed
Balance Exploration and Exploitation Wisely
The impact of multiple epochs in model training underscores the importance of properly managing data repetitions, as increased data weight can lead to improved memorization and potential discovery of new phenomena. However, achieving an optimal trade-off between extensive resource-driven runs and smaller-scale explorations presents challenges. Adopting a first-principles approach is crucial, particularly emphasizing the significance of high-quality data. Employing manual processes, robust analyses, and classifiers enhances data validation. Testing ideas on a smaller scale demonstrates improvements and guides decision-making for teams training large language models.
00:00
Transcript
Episode notes
Get the Snipd
podcast app
Unlock the knowledge in podcasts with the podcast player of the future.
AI-powered
Discover
highlights
Listen to the best highlights from the podcasts you love and dive into the full episode
Save any
moment
Hear something you like? Tap your headphones to save it with AI-generated key takeaways
Share
& Export
Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more
AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
Discover
highlights
Listen to the best highlights from the podcasts you love and dive into the full episode