Transformer Memory as a Differentiable Search Index: memorizing thousands of random doc ids works!?
Neural Search Talks — Zeta Alpha
00:00
Is It a T5 Model?
I definitely see the process. I think the intuition is something like if you can find the 10 top salient terms from a document, this will work for any new document you add. But I don't know how this interacts with training or beam search or anything like this also. It's even the ordering of these terms, I'm not really sure what to say about it, especially if you're using teacher forcing. Yeah. And you might have some problems enforcing uniqueness in these terms as you add, right? Because if you add documents to a collection, two documents might be similar in ways that you know. They're no longer, they're not necessarily unique. So yeah, my guess is
Transcript
Play full episode
