MLOps.community cover image

Handling Multi-Terabyte LLM Checkpoints // Simon Karasik // #228

MLOps.community

NOTE

Fine-tuning Models with Fewer GPUs

Fine-tuning a model requires fewer GPUs compared to the initial pre-training phase which may involve using thousands of GPUs. Fine-tuning involves smaller datasets, typically containing only thousands of rows, unlike pre-training which could involve billions of rows. Even though the model size remains the same, the workload changes as the data is now processed by fewer machines instead of a large cluster. This shift in workload impacts the cloud requirements, with a focus on scalability and the ability to handle the load from fewer machines efficiently, rather than distributing it across a large number of machines.

00:00
Transcript
Play full episode

Remember Everything You Learn from Podcasts

Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.
App store bannerPlay store banner