I don't think empathy makes any difference at all, because people will just reinterpret their empathy to support whatever they're doing. Empathy is completely subjective, and it's the last thing I'd want to trust. Here's what I would trust more than empathy. Don't hurt people, or we're going to turn you off forever. Does AI have a self-defense mode? AI has whatever mode is learned by scouring the internet for all human actions. You won't be able to define what hurts people. See, one of the reasons that AI might be limited or banned is that everything AI recommends will hurt people.