The Importance of Long Context Models

There's no reason to believe that if you give a model a long sequence, it will attend to the right things. I'm honestly really excited about replet applications for these long context models specifically because you're going to get a lot of user feedback very quickly. You'll see the rates go up depending on whether you have a longer context.