101 - The lottery ticket hypothesis, with Jonathan Frankle
NLP Highlights
00:00
Is There a Lenar Path Between a Network and a Pox?
There is a lenar path between the network after it's been trained to completion and at the state where it's been training only for k pox. If you have a network at a particular iteration k, and you train two copies of it under different samples of stedy noise, those copies will always find a minimum in which they are linearly connected. The easiest way to get an or network to learn effectively is to start over bramaterized. But with that set i think there's a lot of nuante here. It would be very nice if we knew how to find them right then and there. And the holy grail is an approach that allows us prune early in
Transcript
Play full episode
