Programming Throwdown cover image

153: ChatGPT

Programming Throwdown

00:00

The Attention Is All You Need

GPT 3 is basically that forward language model joined with a query language model. It embeds that query you made into a certain space and then it uses that space to start generating content. And so what you really want is not just the most probable word at every step, right? What youreally want is a string of words where the sum or maybe the product of those probabilities is really high. So there's all these tricks. When you layer them all together, you end up with something extraordinarily powerful.

Transcript
Play full episode

The AI-powered Podcast Player

Save insights by tapping your headphones, chat with episodes, discover the best highlights - and more!
App store bannerPlay store banner
Get the app