How Does GPT Look Like a Transformer?

Alec Bradford is the first author on GPT-1 to clip. He's done incredible work, considered to be the most successful independent individual contributor at OpenAI. One intuition that he gave me was you have to think about all these large generative models as distillation of human activity on the internet baked into a large model. You're basically leveraging human activity like rating pages, creating content, describing articles and writing it in a formal way.