AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
GPT-3: A Black Box Meta Learning Method
The GPT-3 has this behavior where you can now give it a few examples in its context and it quickly learns the task that you encoded in the context. It almost seems like it's acquired yeah it somehow learned to learn just through this language modeling pre-training. The failure cases are quite illuminating and makes it pretty clear at least to me that we have a lot of work to do on the mission they're in front beyond just feeding a lot of data into the systems.