AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
Is GPT 12 a Scaled Up Version of GPT Two?
You can turn a lossy compression into a lossless compressor pretty easily using an arithmetic encoder. You don't have to know whether it's an E or an I, you just have to put good probabilities on them and then code those. But no, we're not just going to be able to scale up to GPT 12 and get general purpose intelligence with something as dumb as this kind of loss function.