The Power of Reducible Loss in Image Net Classification

When generative image models are fine-tuned, you find a power law for classification loss versus model size. As you increase by 10, so you go from million to billion to trillion, you get the same return every time,. Which maybe is bad because once you're going to billion, it's very hard to get to trillion to get another 10 percent.