Eric Michaud on scaling, grokking and quantum interpretability
The Inside View
00:00
The Messier Scaling Exponents of Open AI in the Wild
So are you saying that the maybe the experiments in Chile, don't fit with your like risk payments or your theory? No, so I guess like we would expect that like the data scaling exponent is going to be less than the parameter scaling exponent. And this is like indeed what they observed in like the Chinchilla scaling law studies. So it seems like somewhat encouraging but if you actually look at like the precise value of like what these like scaling exponents are it's sort of like not exactly what we would predict.
Transcript
Play full episode
