Transformer Memory as a Differentiable Search Index: memorizing thousands of random doc ids works!?
Neural Search Talks — Zeta Alpha
00:00
Is T5 a Better Encoder?
My sense was that maybe it could be optimized a bit more trying to compare natural questions numbers from different places, but this is always really hard to compare across papers. So I wonder how that... Yeah, I guess we could say with Burt-based encoders and other encoders, it is very important to do this hard negative mining or it seems to be important. There's different ways to do it, but it always improves. And based on the fact they're not doing it here, we either have to say that this could be improved or for some reason it's not needed with T5,. But again, this is speculation. It makes sense.
Transcript
Play full episode
