The Importance of Context in Machine Learning

In the field of machine learning, at least in the industrial setting, I'm in sort of things are much have become much more of a software engineering culture. So you don't need to understand any of that the transformer is doing it all the heavy lifting right so the engineering has supplanted for the domain knowledge. Now, maybe we should talk a little about the famous attention is all you need paper and try to get a sense for what a transformer is which which I don't think I understand. But didn't they know that before 2018 or not? And didn't they, in some sense, appreciate the importance of context or not?"