Transformer Memory as a Differentiable Search Index: memorizing thousands of random doc ids works!?
Neural Search Talks — Zeta Alpha
00:00
Is There a Constant Storage Cost Every Time You Add a Document?
It's unclear to me how much this improves things. We have to store with every document, assuming they're not semantically meaningful. Also, the doc ideas have to be stored. I'm not entirely sure how to make sense of that. So, maybe it's not a constant storage cost every time you add a document. Maybe at some point, it knows most of Wikipedia. And so, you add this new news event and it's not storing something entirely new. It can be stored in terms of other things that have been stored in some way.
Transcript
Play full episode
