RL challenges and the importance of architecture selection in large-scale offline RL

In RL, there are two common pitfalls: overfitting to target values and discarding too much detail./nAdding data diversity and using larger models can improve RL performance./nSelecting architectures that are easy to optimize and going slightly larger in size can mitigate some of the difficulties in large scale RL efforts./nReinforcement learning requires representation of both optimal and sub-optimal behaviors, with the latter often being more complex.