The Results of the JatchPT Modeling Experiment

Human participants who used the model were significantly more accurate than people under the same constraints. We also noticed, and this is not in the version of the paper that's out yet but we should have a follow up soon, this clearly gets better as the models get better. But the problem is still very much open. The sort of people we're working with are still, these questions are still hard, they're still answering them with sort of less than 80% accuracy.