The Cost of Having a Larger Context Window in LLMs

TPUs weren't that great at RNNs, so they designed kind of a different parallel architecture. We're also looking for whether rapid because we're going to be keep training models based on that framework. So I'm kind of excited what we'll do here. Yeah, I agree. Like, new hardware is unlocking new explorations of architectures. And I guess you guys are the first one to be able to do these abditional studies. But it comes at a cost. There's a cost, but honestly, the cost isn't that big.