AXRP - the AI X-risk Research Podcast cover image

19 - Mechanistic Interpretability with Neel Nanda

AXRP - the AI X-risk Research Podcast

CHAPTER

Yep and That's the Basis of This Algorithm

The model spontaneously forms five to six sub modules for different angles where each angle is some multiple of two pi of n. The neurons spontaneously cluster into a cluster per frequency yep and the output logits are the sum of a cause a plus b minus c times frequency term for each of the five frequencies  the frequencies chosen are basically arbitrary but there's a you do it sort of at different speeds of moving around the circle right?

00:00
Transcript
Play full episode

Remember Everything You Learn from Podcasts

Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.
App store bannerPlay store banner