Scaling in Machine Learning?

We're trying to build some predictive scaling for our a pridow. One of the problems with allenges of a start up that's doing mchi ording in production is, you know, we deploy like hundreds of gpus and thousands of c p cores inte production every day. And if you overscale that, like size, then there's just huge cost tho come with that. If you underscale, there's bad performance for en users.