Is It a Technical Mistake or an Ethical Mistake?

Consider a programme that is learning to make decisions about the amount of parole some one should have. We might think it's inadmissible to take someone's race or gender into account when making a decision like this. But if it's statistically correlated, and maybe for complex sociological reasons, statistically correlated with racitivism, repeat offence, then the machine will use that information. That's not a technical mistake, because it is predicting accurately using that information. But we might Think it's a moral mistake, it's an injustice. One model of how we may respond is, well, maybe we could build in some ethics, and certainly you could build into machines prohibitions on using certain data.