Orca: Progressive Learning from Complex Explanation Traces of GPT-4
Deep Papers
00:00
The Differences Between Chat and GPT for Data Sets
One thing that I thought was kind of interesting I don't know if they talk about this a little bit later, I think they actually talked about this in the next section or the next page. They got slightly better performance by first training at all on chat GPT and then later on GPT for the idea being there's some kind of curriculum learning thing going on where you learn from the simpler model. And so, you know, it's kind of useful in kind of the fine tuning data selection like we clearly more thoughtful than,you know, then the first versions.
Transcript
Play full episode
