Transformer Memory as a Differentiable Search Index: memorizing thousands of random doc ids works!?
Neural Search Talks — Zeta Alpha
00:00
Zero Shot Transfer
It's still working to some amount. So maybe it's not fair to say it's BM25 territory, but they've lost something like half of the gains. I think that I'm just now realizing that I don't quite understand these. Why would the results be better on the larger data set? Shouldn't they be monotonically harder as there's more documents? There are more documents, but the queries and documents are both changing. It might be interesting to have like a minimal query subset evaluated in all three.
Transcript
Play full episode
