RL Algorithm Design and Reward Design

Within the scope of developing reinforcement learning or RL algorithms, the goal is to build agents that learn to achieve the given objective. Specification gaming involves an agent exploiting a loophole in the specification at the expense of the intended outcome. These behaviours are caused by miss specification of the intended task rather than any floor in the RL algorithm. Even for a slight misspecification, a very good RL algorithm might be able to find an intricate solution that is quite different from the intended solution. This means that correctly specifying intent can become more important for achieving the desired outcome as RL algorithms improve. It will therefore be essential that the ability of researchers to correctly specify tasks keeps up with the ability of agents to find novel