Large models on CPUs (Practical AI #221)
Changelog Master Feed
00:00
How to Optimize a Large Language Model
The biggest gap in terms of trying to get people off of GPUs on a CPU is the model optimization that needs to take place first. That's what we've been also specializing on at neural magic, making all the research that we're doing into open source and building out a SaaS platform so everyone can easily play with high parameters. And I'll take two steps to doing that. One is just covering kind of the 90, 95% class, at least where we've been able to get to on those. The second is looking specifically at large language models.
Transcript
Play full episode
