The Importance of Good Calibration in AI Performance

Open AI has released a paper on how well calibrated its text autocomplete model is. The study looked at the raw model GPT 4 and compared it to one that was fine-tuneed with human feedback for conversational effect. It found that the fine-t tune version, although more human-like actually is worse at being calibrated. "It's not clear if they're necessarily at odds but they certainly have diverged in this particular context yeah"