AXRP - the AI X-risk Research Podcast cover image

31 - Singular Learning Theory with Daniel Murfet

AXRP - the AI X-risk Research Podcast

CHAPTER

Phase Transitions in Neural Networks

This chapter examines the concept of phase transitions in machine learning, especially in relation to large language model training. The discussion reveals uncertainties when applying theoretical predictions to larger models, emphasizing the need for deeper analysis. It also introduces the role of weight-restricted local learning coefficients and how they relate to Bayesian phase transitions, bridging statistical models with physics concepts.

00:00
Transcript
Play full episode

Remember Everything You Learn from Podcasts

Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.
App store bannerPlay store banner