The Evolution of Neural Language Models

I'm curious, how do you see the evolution that we have seen play out over from your perspective? The last five ish years where language models have gone from something most people don't even pay attention to in early days. I think there's a big shift with attention that really changed a game. We train on larger data sets and we have these bigger models. And it makes a huge difference. It tells me a lot about what we could do with large neural nets. This is one of the most important discoveries of recent years.