161: Leveraging Generative AI Models with Hagay Lupesko
Programming Throwdown
00:00
How to Fine Tune a Model for Screenwriting
The idea is that you can do fine tuning much more efficiently by like decomposing matrices. And then your fine tuning is more efficient, but then you can also apply. You can take a base model and apply fine tuning by just, you know, applying the factorized matrays you got from Laura. The other thing I would say is that it's also much cheaper and faster than the pre-training. It was like we recently open sourced a model called MPT7B, so 7 billion parameters. It was trained on one trillion tokens of text, of language, which is huge.
Transcript
Play full episode
