Lost in the Middle: How Language Models Use Long Contexts
Deep Papers
00:00
The Open Source Encoder-Decoder Models
Sally Ann: I think scientists are going to kind of dive into, like, the why behind all of this. Amber: We're really glad this paper came out because it's making us reevaluate where we're spending our time and priorities. Sally Ann: It's such a straightforward thing. The way that you can just play around with like the positions and actually see how things are doing is very interesting. But obviously model performance is highest when the relative information occurs at the beginning or end of the input context. A lot of people are focusing on, but you'll see Sally Ann and I are going to be focusing on a few other concepts that this paper really showed.
Transcript
Play full episode
