How Do You Best Use Human Supervision for Learning and Specification?

When you think about this problem, it seems like the question of what should the reward be is also a question of what is the environment and what are the tasks. How do you best use human supervision both for learning and for specifying is something that's very interesting as well. My own views are not very rigid onlike what will win out here. It's more likely that things that are easy to specify and like scale well.