How to Train a Neural Network Smaller

It's an expensive proposition to run a research group doing neural networks. So wouldn't it be exciting if, you know, we could just train smaller networks? Sure. A lot of people see the pruning results and they say, well, doesn't that mean we can just training smaller networks from the beginning? And the answer is no.