Razib Khan's Unsupervised Learning cover image

Nikolai Yakovenko: GPT-3 and the rise of the thinking machines

Razib Khan's Unsupervised Learning

00:00

Is GPT-3 Just the Beginning?

GPT-3 has 175 billion parameters, but the Megatron model from NVIDIA already did a trillion. Microsoft is powering both GPT-3 and sort of the other great giant model for my former team that's still under development. I mean, once you know how it works, you're absolutely right, recreating it is like not a big deal. For text, you need a lot more parameters because you do kind of need to compress the knowledge of the internet.

Transcript
Play full episode

The AI-powered Podcast Player

Save insights by tapping your headphones, chat with episodes, discover the best highlights - and more!
App store bannerPlay store banner
Get the app