The Importance of Large Language Models

With 200k, they kind of get into the vicinity of what would be a reasonable explanation, but they don't get it right. But nevertheless, I'm grateful that we're seeing models with larger context windows because the only way to find out is to empirically test LMS. It's possible just break the documents into different chunks. And then you have another model to take on the embeddings of different chunks and generate the embedding of the whole document. So for today, language model mostly is just like generation conditions on the embedDings of, like, of the raw text, not the joint embedding. Does that make sense? Yeah.