Large models on CPUs (Practical AI #221)
Changelog Master Feed
00:00
The Cost of Deployment on GPUs and CPUs
When people get into larger deployments on ML and neural networks, the cost significantly shifts. You're for a lot of larger enterprises that are actively deploying 80, 90% of their costs is purely in deployment on these machines. So there can be a significant reduction once you're at that scale. As soon as you go into deployment, model optimization is a great thing to start because it's essentially just free performance that's left on the table That can significantly affect your bottom line.
Transcript
Play full episode
