Is That Mouse a Cookie Story?

As neural networks get bigger, they become qualitatively more and more impressive. There's no idea from the engineering community that really made all of that possible. It's really kind of what the model itself is discovering in the data. So when a language model is trying to predict the next word, if it's maybe read a few paragraphs of a story on a fanfiction website or it's read a fewparagraphs of a blog or something like this, it's going to try to figure out who’s talking,. What did they believe? And use that kind of reasoning as part of predicting the next word.