Retrieval Algorithms

Realm stands for retrieval augmented language model. The basic idea behind that is, so for people who are familiar with things like bert the the overall game you're playing there is you take text that is unlabelled and you mask out some words in the text. And oftentimes, if you mask out certain things, they can be actually very world knowledge heavy. In this way, you are basically accessing this external knowledge base, as opposed to incoding all this knowledge into the various ways of your knowing it,. Right?