So how was this different from sort of classical linguistics? You know, because I remember in sort of the earlier days of AI, seeing dozens, if not hundreds of dissertations in natural language processing, there was a prominent NLP researcher Roger Shank at Yale who passed away recently, May he rest in peace, who supervised dozens of dissertations in NLP. And I don't know whether it would be unkind of me to say this, but a lot of that stuff seemed to come to not much that at the end of the day, it didn't seem to work. So what was the turning point that sort of made started making things work?

Yeah, so there's sort of two turning points. I think some of the context that had happened before a lot of what I was describing kind of set the stage was what's called the statistical revolution NLP. And the idea there was kind of was to give up, at least temporarily, on this vision of kind of fully writing down, fully manually implementing linguistics and cognitive science. I think a lot of kind of the first generation or generations of NLP, including a lot of Shanks work, I think was built around this vision that we would figure out how language and reasoning works. We'd write that down and the computer would do it. And there's this, I think, sort of gradual realization throughout the 80s and 90s that there was just too much there that we could make this work in very limited targeted settings. But that sort of human reasoning was sort of too heterogenous and diverse and complex for even a thousand people for 10 years to be a big enough team to implement it all. And so the statistical

revolution

kind of shifted toward this orientation of let's learn everything from data. Let's build models that are guided somewhat by our intuitions, but ultimately fill in a lot of the blanks by trying to infer what humans are doing from big, usually labeled datasets. And that was, I think, what got a lot of early NLP off the ground. This is the point where sort of the first machine translation system started getting meaningfully deployed. But so, yeah, I think that idea was historically quite important and sort of reoriented the field around learning. But up to that point, I think the actual tools we had for doing the learning were pretty weak, that there was a lot of interest in linear models and in models where kind of a lot of the work still lived in these feature functions. A lot of the work was still kind of setting up the model very precisely according to your understanding of what it was supposed to be doing such that some learning could take place around the edges. And that still sort of hit the same basic problem that just does a lot of complexity there and it's more than we could really write down. And so I think the big, I think the sort of second big shift that this early neural networks work was gesturing towards and that kind of did play out over the following decade was toward really fully embracing this learning paradigm, fully embracing this idea that our job as machine learning researchers, computer science researchers applied computational linguists was just to figure out how to get the machine to learn, not to tell it anything we already knew within the hope that it would then pick up all of this complexity and nuance if we gave it just enough data and

enough time.