Transformer Memory as a Differentiable Search Index: memorizing thousands of random doc ids works!?
Neural Search Talks — Zeta Alpha
00:00
Is the Failure Modes a Problem for Dual Encoders?
In terms of failure modes, I think it's not necessarily different. The failure modes that exist for dual encoders I think also exist here. It doesn't seem like a very efficient way of storing things. Do you think that this is solvable by some optimizations like pruning, compensation of these sort of techniques? Or do you think maybe this is more of a fundamental efficiency problem of this approach more fundamentally that we make it hard to?
Transcript
Play full episode
