Transformer Memory as a Differentiable Search Index: memorizing thousands of random doc ids works!?
Neural Search Talks — Zeta Alpha
00:00
Is There an Index in a Transformer Model?
The way I think of it is doing some implicit entity typing with the sentence. So it guesses this entity is a football club or a person or something. And then maybe some usage of patterns in order to come up with, you know, the right canonical name. The interesting part is that it's all implicit, right? There's no component that is like type Chelsea classified as it's all like ultra aggressive generation.
Transcript
Play full episode
