Machine Learning Street Talk (MLST) cover image

Neel Nanda - Mechanistic Interpretability

Machine Learning Street Talk (MLST)

00:00

Exploring Complexity in Language Models

This chapter discusses the implications of larger model and data sizes in language models and their ability to perform complex tasks, such as coding and solving logic puzzles. The conversation examines whether these models memorize data or develop a deeper understanding of language and world dynamics, highlighting insights from the Othello paper and exploring the intricacies of neural network behavior and interpretability. It critiques existing approaches and theorizes about the nature of AI reasoning, emphasizing the challenges of model learning and representation.

Transcript
Play full episode

The AI-powered Podcast Player

Save insights by tapping your headphones, chat with episodes, discover the best highlights - and more!
App store bannerPlay store banner
Get the app