AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
Scaling Theory
#8 – Sara Hooker: Big AI, The Compute Frenzy, and Grumpy Models
Aug 5, 2024
Sara Hooker, VP of Research at Cohere and a recognized AI innovator, shares insights on scaling laws and their limits, emphasizing how smaller models can outperform larger ones. She discusses the balance between open-source and proprietary models, highlighting the need for inclusivity, particularly for multilingual capabilities. Sara also tackles data accessibility challenges and copyright issues affecting AI training, and reflects on how her diverse upbringing informs her approach to innovative research practices. Expect a thought-provoking conversation on AI's future!
55:14
Episode guests
AI Summary
AI Chapters
Episode notes
Podcast summary created with Snipd AI
Quick takeaways
- Scaling laws in AI are complex, revealing that model performance often depends more on optimization choices and data quality than on sheer computational power.
- Openness in AI research enhances innovation and multilingual advancements, highlighting the importance of collaboration and diverse input for optimal solutions.
Deep dives
Scaling Laws and Their Complexities
The relationship between compute capacity and AI performance is nuanced, challenging the common belief that increasing size and complexity of models automatically leads to better outcomes. Recent discussions highlight instances where smaller models have outperformed larger counterparts, such as an 8 billion parameter model outperforming a 176 billion parameter model. This suggests that optimization choices and the quality of data may play a more crucial role in model performance than sheer computational power. Thus, it becomes apparent that scaling laws may not be as straightforward or linear as initially perceived, requiring a deeper understanding of their implications in AI development.
Get the Snipd
podcast app
Unlock the knowledge in podcasts with the podcast player of the future.
AI-powered
Discover
highlights
Listen to the best highlights from the podcasts you love and dive into the full episode
Save any
moment
Hear something you like? Tap your headphones to save it with AI-generated key takeaways
Share
& Export
Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more
AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
Discover
highlights
Listen to the best highlights from the podcasts you love and dive into the full episode