What's GPT-3 Doing for Open AI?

Speaker 2
And I believe open AI requires attribution in principle for text generated by their system. So it's sort of interesting, like there's a whole philosophical hairball there we could get into, but like, you know, GPT-3 is trained on text from the internet. And then it's used to generate text. And apparently that connection, like you need to give it credit for that. But what about the text that it used to look like? It's very convoluted and depending on how you think about what GPT-3 is doing, different things can make sense. But generally like, I'm wondering, how did this change your perspective? How did it change your perspective on what language models may be able to do in the future? Were you disappointed or were you impressed by what GPT-3 was able to do relative to your expectations going in?
Speaker 1
Yeah, absolutely. And you made a really great point earlier about like how, you know, companies are just putting wrappers around this. And I think the fact that, you know, like we can build this in the weekend, meanwhile, companies are like selling this for, you know, really high prices. For me, that was like, I don't know how I felt about that. On the one hand, I was like, okay, interesting, we can like go build a company and maybe if we like created like a fine-tune better model, right, we can go and like do that. But on the other hand, I was like, on the ethical issue for those people, for those of us who aren't as interested in like understanding like the nuts and bolts of GPT-3 or like, we don't even have like the opportunity to do that. Like those of us who aren't like in the field, for example, isn't there like an ethical issue for like pricing something so high when like in reality, most of what you're using is kind of open source or like pretty cheap and mostly like from my understanding, the main purpose was like for research, right? So that's an interesting perspective when we like productionize and also to an extent like incentivize firms and other private companies to create a business model around this, a lucrative one at that given how many different competitors are popping up and how prominent these competitors are not just in the US, but in different countries as well.

129. Amber Teng - Building apps with a new generation of language models

Towards Data Science

Remember Everything You Learn from Podcasts