Transformer Memory as a Differentiable Search Index: memorizing thousands of random doc ids works!?
Neural Search Talks — Zeta Alpha
00:00
Is There a Differentiable Search Index?
In the context of this paper, they use this differentiable search index. Can you like give an explanation of why that term makes sense or why they call it this? Yeah, how I understand the term is they're essentially saying they're storing documents in the transformer. And so now the transformer becomes an index in some sense. So I guess it's saying the model weights themselves are where the documents are now stored.
Transcript
Play full episode
