Do Language Models Know the Way to Rome?

This paper examines the extent to which language models can represent geographic knowledge, such as understanding the relationship between city and country names.

It finds that larger models perform better in encoding geographic information, which can be inferred from higher-order co-occurrence statistics in their training data.