
31 - Singular Learning Theory with Daniel Murfet
AXRP - the AI X-risk Research Podcast
Phase Transitions in Neural Networks
This chapter examines the concept of phase transitions in machine learning, especially in relation to large language model training. The discussion reveals uncertainties when applying theoretical predictions to larger models, emphasizing the need for deeper analysis. It also introduces the role of weight-restricted local learning coefficients and how they relate to Bayesian phase transitions, bridging statistical models with physics concepts.
00:00
Transcript
Play full episode
Remember Everything You Learn from Podcasts
Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.