How to Fine-Tune a Model to Get the Results You Want

The original models, as I understand it, since you were trained on text on the internet. You collect some text from the internet and train a next word or a next token prediction model. And from that it learns how to generate responses and completions - so forth. Now, the challenge, of course, is that not all text on the web is high quality or even maybe not everybody wants the same thing and so forth. So first you train a huge model on ton of web pages straight from the web, hopefully cleaned up a bit. But the next thing you do is fine tune that model on an extremely curated, very refined data set.