Eric Michaud on scaling, grokking and quantum interpretability
The Inside View
00:00
The Functional Unit of a Deep Neural Network
In your paper with Quanta, you talk about things being discrete. So I'm kind of curious, what do you mean by discrete in these kind of scenarios? When everything is floating point, I really think. Ah, yeah, that's a great question. Yeah, so I guess maybe there are a couple high level models you could have for thinking about what neural networks are doing and what they consist of. Maybe an example of something that feels more discrete, an algorithm in the network is the circuit that does induction.
Transcript
Play full episode
