The Binding Constraint in Deep Learning

At the time we didn't really have the compute power. So you're building this model and saying, okay, now to learn what you need to learn, read literally every word on the internet. And so then we tried it and we found out actually this thing is pretty compelling. It can understand things that our models previously have never understood,. But I will say the second, and this gets to the second hard part. Once we had these large scale language models, we didn't quite know how to put them into search ranking. This was not something that had been done before.