How to Make Language Models Have Superhuman Performance

Language models, such as GPT-3, can surpass human performance on certain tasks when prompted in a specific way. In a study, the oldest and smallest GPT-3 model from 2019 only achieved a 57% accuracy on a yes/no question set, while humans scored 95%. However, when the prompt was adjusted to provide more relevant examples, the language model achieved a 94.5% accuracy, surpassing human performance. This suggests that language models can be trained to have superhuman capabilities.