The Importance of Reducible Loss

Even if you had the perfect model that could perfectly learn everything from a training set, it can't get to zero loss because there's some amount of randomness. And so you can find to some extent this irreducible loss and that becomes a constant offset. So perhaps what's really important metric that we should focus on is the component of the loss that can be learned, so the reducible loss.