#8: Tamay Besiroglu on the Trends Driving Past and Future AI Progress
Center for AI Policy Podcast
00:00
Exploring the Relationship Between Data, Compute, and Model Scaling in AI Training
The chapter explores the importance of scaling data and model size proportionally with compute when training AI models, with a focus on recent attempts to replicate a paper on correct scaling practices. It emphasizes the errors in estimating parametric scaling laws and the significance of accurately predicting the model's loss value based on size and training tokens.
Transcript
Play full episode
