The Importance of Language Models in Review Processes

GPD4 performed much better than all the other language models out there and we tested quite a few including some new ones or relatively new ones at the time I would say. The first study that we did was we checked if GPD4 or like large language models in general could find errors that we inserted into shortened versions of papers. Third experiment which was just having a naive version of a full blown review where you ask the large language model whether paper A or paper B is better. We would ask it to evaluate the submissions based on actual scientific contributions so here the outcome was quite objective.