The Evolution of GPT

The GPT algorithm is almost identical to the predictive text algorithms that you see in your email account or on your smartphone when you're typing. When it looked at these stories, fairy tales that begin once upon a time there was a young higher probability for girl and boy and you see that skewed in the data. There's an additional factor to keep in mind which is that the probability tables that it builds don't really build probabilities but they emit logits that are then interpreted as probabilities. They're very, very conditional on the prior text and the structure of the of the sentence that it's currently completing.