
19 - Mechanistic Interpretability with Neel Nanda
AXRP - the AI X-risk Research Podcast
Yep and That's the Basis of This Algorithm
The model spontaneously forms five to six sub modules for different angles where each angle is some multiple of two pi of n. The neurons spontaneously cluster into a cluster per frequency yep and the output logits are the sum of a cause a plus b minus c times frequency term for each of the five frequencies the frequencies chosen are basically arbitrary but there's a you do it sort of at different speeds of moving around the circle right?
00:00
Transcript
Play full episode
Remember Everything You Learn from Podcasts
Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.