How to Reduce the Cost of Training a Large Model

Aims to reduce the compute cost of training a large model by 40%, 50%. Initial results on billion dollar models, GPT two in particular. It's really interesting because this is one of those things that fundamentally changes how we think about the process of training a model. So it's kind of interesting to see that get shaken up.