Deep Learning Systems in the Wild?

I think there's a tendency right now to measure a model's performance as an aggregate metric, whether it's mean average precision or what have you. But where I see the gap between academia and industry there is that in industry, people are increasingly measuring not just aggregate performance, but kind of curated unit tests almost. Of the subset of my data that looks like this weird edge case, how did we do? And I think collectively there, you can still get an interpretable glimpse of where your model's doing well or it's not doing well, even if you don't know exactly why.