3min snip

INSIGHT

Post-training Impact on LLMs

John Schulman wasn't fully convinced of the revolutionary potential of LLMs until after GPT-3.
After GPT-3's success, the focus shifted to leveraging language models.
Post-training methods yield significant improvements, even surpassing web data quality in model output.
GPT-4's Elo score increased significantly (100 points) primarily due to post-training enhancements.
Several factors, like data quality, iterations, and annotation changes contribute to this compute increase.

00:00

Transcript

Episode notes

Get the Snipd
podcast app

Unlock the knowledge in podcasts with the podcast player of the future.

Listen to all your favourite podcasts with AI-powered features

Listen to the best highlights from the podcasts you love and dive into the full episode

Hear something you like? Tap your headphones to save it with AI-generated key takeaways

Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more

Listen to all your favourite podcasts with AI-powered features

Listen to the best highlights from the podcasts you love and dive into the full episode