The Role of Counterfactual Inference in Policy Debates

In the ARPA tournament to date, what we've asked conditional questions. So if you as policy goes toward A or B, how likely is a given consequence? Now, this allows us to assess the accuracy in the world in which one of the other policies is implemented. We don't know what would have happened if we'd done policy B. That's in the realm of counterfactual. This is one of the great limitations in our ability to learn to become better calibrated in historical environment.