Randomize Modules in a Neural Network

The data comes in at the left-hand side here and that's kind of where the sort of noise is generated. And we see that for the most part, there's no obvious correlation between where in the network the module is and what effect zerowing out will have. Another thing we can do is just to progressively mix randomness uniformly into every weight in the network. It's kind of like globally drugging the brain.