Transformer Memory as a Differentiable Search Index: memorizing thousands of random doc ids works!?
Neural Search Talks — Zeta Alpha
00:00
Using T5 Models for a Natural Question
This is unsurprisingly a natural question that's related to a Wikipedia passage. And then there's one relevant document for that question. This is crowd sourced where I think crowd sorcerers and crowd workers came up with the questions. They're starting from pre-trained T5 models, you know, going up in size from the base model to the extra-extra-large model. So like dense retrieval, yeah.
Transcript
Play full episode
