Ep 7: Co-Creator of Databricks Dolly Mike Conover on Open-Source LLMs
Unsupervised Learning
00:00
The Importance of Labeled Data in Chat GPT
Dolly v1 can write letters in a well formatted template. It understands salutations. The Dolly v2 does not, it can write letter like pros, but it doesn't understand what the beginning of a letter looks like. And so I think that is because our annotation rubric was, like I said, Google form,. You know, there's kind of like how, in part, you generally pay annotators to do this work. But we have started to, there's some function that is not well understood, or at least the people that understand it are not talking about it. We're just nudging the flywheel rather than requiring humans to think up everything that model knows
Transcript
Play full episode
