#66 – Michael Cohen on Input Tampering in Advanced RL Agents
Hear This Idea
00:00
The Problem With Reward Modeling in AI
For certain narrow tasks, you could probably come up with a policy that does that task well without doing explicit on the fly reasoning about what it needs to be doing. But for AI that's able to act generally in the world and for some economically useful tasks, you really do need to be able to learn different things to do as they come up. And once you have a reinforcement learner that has to model the world in quite a bit of detail anyway, if you have it doing tasks that really require understanding computers and cameras,. It no longer becomes very expensive to represent the hypothesis that reward is determined by the number the camera sees rather than the number the box displays.
Play episode from 45:10
Transcript
