AXRP - the AI X-risk Research Podcast cover image

38.2 - Jesse Hoogland on Singular Learning Theory

AXRP - the AI X-risk Research Podcast

CHAPTER

Understanding Model Complexity in SLT

This chapter examines model complexity through the lens of Singular Learning Theory, focusing on the local learning coefficient and its relationship with memorization across various tasks. Additionally, it introduces the refined restricted learning coefficient and the concept of a multigram circuit, illuminating how attention-based models process complex language structures.

00:00
Transcript
Play full episode

Remember Everything You Learn from Podcasts

Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.
App store bannerPlay store banner