Episode 20: Hattie Zhou, Mila, on supermasks, iterative learning, and fortuitous forgetting
Generally Intelligent
The Story of Coherent Gradients
It's not clear that high gradient coherence really should lead to better generalization, either. I think the story of coherent gradients is very simple. It makes sense, but it's probably not the full story. But we don't know if that's harmful either. Could there be a way that that's beneficial? It's possible.
00:00
Transcript
Play full episode
Remember Everything You Learn from Podcasts
Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.
