No Press Diplomacy - Can You Model Humans Better?

We started off just doing equilibrium computation with just a value function trained on human data. And that actually worked out pretty well, but it was clear that was not going to get us the superhuman. One of the major lessons to learn from the no press work is, again, the significance of search. We took a very planning based approach, a very search based approach to the game instead of focusing so much on reinforcement learning. That ended up working out quite well for us.