The Secret Sauce That Made All That Fast

The company is focused on helping people actually train their own like LMs that are useful for them. A lot of what I work on nowadays is trying to optimize not just the training efficiency of our stack, but also how many dollars it would take to train a 70 billion prime model. The answer is 5% here, 5% there. It's really blood, sweat and tears of just an incremental improvement and incremental improvement and it adds up really quickly to get to these huge numbers.