The Importance of Benchmarks in Natural Language Processing

The language balls are trained on the internet, basically. We think it's kind of an indication, right? So you can safely assume that at least when it comes to common entities, like not those from the tail of the distribution, the model knows a lot of things about these entities. These benchmarks really help to figure out what these models are capable of and what they fail at which, you know, drives progress and interesting insights.