The Complexity of Large Language Models

The amazing thing about these algorithms is that they typically are very simple minded and yet they they work very well. One reason is that with a large enough data set, which is what happens when you crawl much of the Internet is that the statistics sort of work in your favor. The premise is that words which are closer have closer semantic meeting in words which are far apart. And that already buys you a lot of mileage. These transformals are much much much more complicated it's not at all. But by analogy you can think of what they're doing as some kind of let's say glorified or to VEC.