The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence) cover image

Is It Time to Rethink LLM Pre-Training? with Aditi Raghunathan - #747

The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)

00:00

Knowledge Localization in Machine Learning

This chapter explores the innovative concept of knowledge localization in machine learning, emphasizing the role of training objectives in parameter updates. It introduces 'memorization sinks' and discusses how specific neurons can be organized to balance localized and shared knowledge within large language models.

Transcript
Play full episode

The AI-powered Podcast Player

Save insights by tapping your headphones, chat with episodes, discover the best highlights - and more!
App store bannerPlay store banner
Get the app