The Limits of Common Sense

GPT 4 fails to answer three questions that give a hard time to intelligence. The more you do RLHF with these failure examples, the better performance you will get,. But the question is, how do you know for sure that you covered everything? We will never know.