Optimizing Deep Learning Models

It really depends how deep you want to go, I guess. You don't have to worry about a lot of this because of libraries like PyTorch and Lightning where we handle it under the hood. But if you really want to optimize deep learning and it depends on the skill, I'm talkingskill of like hundreds of GPUs or thousands,. If you're talking about single GPU or a few, I think that's very straightforward to optimize. To optimize something like multi-node training where you have thousands ofGPUs is another matter. And then once you have the data on the machines, then you have to optimize literally what actually the models are doing. It gets really, really performance heavy