AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
The Importance of Absolute Performance in Language Models
I had another question about the absolute performance of a model and yeah let me unpack this a bit so if we go back to what you mentioned about the APIs then we might have a really powerful API that we can access but we actually don't really know what's going on behind the scenes. So as a research community is it okay to not worry about the absolute number and say like well we understand this one better? Is that kind of an okay state of affairs to have for research if that makes sense?""Yeah I think it's absolutely okay," he said. "You want actually way more of the understanding and careful study than you want of the like chasing the state-of-the