The Relationship Between Reducible and Irreducible Loss

The relationship between the loss and compute or model size is a power law plus constant offset fit. The value you're approaching as you go to infinite data, infinite compute, infinite model size is sort of the true entropy of the data you're trying to model. And maybe now we can dive into a bit of the details. So you also have, in describing results, a pretty interesting idea of reducible and irreducible loss.