The Impact of Chat GPD on User Expectations

The chat GPD will be released in November. It's a very seductive place to be because you have all of these state of the art models available for free download. But teams that go ahead and they drop it into a system like haystack or gene AI, that's exactly the problem they run into. Introducing the embedding is taking 500 milliseconds. And that's a huge problem. There's a lot of effort and energy that goes into getting smaller and distilled versions of these large neural networks that can run with acceptable latencies on production setting.