Razib Khan's Unsupervised Learning cover image

Nikolai Yakovenko: GPT-3 and the rise of the thinking machines

Razib Khan's Unsupervised Learning

00:00

The GPT Three Is Just a Stylized Factor, Right?

The GPT three is kind of brute forcing it. It has some intuitions that it develops through brute force methods. So for example, you get something called mode collapse where if I asked it to write a poem and then I'm like, nah, rewrite that poem. Like the new poem might get us like a little bit different, but not very different. That is really just a structure of how it generates because remember it's generating it like QPT-3,. A word at the time, with this sort of like, indirect rewardwhere it's trying to guess what would please my master?

Transcript
Play full episode

The AI-powered Podcast Player

Save insights by tapping your headphones, chat with episodes, discover the best highlights - and more!
App store bannerPlay store banner
Get the app