How to Get an Extremely Adversarial Initialization

Hard examples are very important for better generalization, especially to weird examples. If your test example is a very common one, you actually, it's easy to get it right but when it's hard, that's when you need the hard examples. In this RAN labels example that you gave is the problem set up such that you ran a plain label like let's say you take image net and then you give all the images random labels and then the model learns to. So the model basically learns really weird features. That's one way to get an adversarial initialization essentially.