The Art and Challenges of Model Training

Karina Nguyen leads research at OpenAI, where she’s been pivotal in developing groundbreaking products like Canvas, Tasks, and the o1 language model. Before OpenAI, Karina was at Anthropic, where she led post-training and evaluation work for Claude 3 models, created a document upload feature with 100,000 context windows, and contributed to numerous other innovations. With experience as an engineer at the New York Times and as a designer at Dropbox and Square, Karina has a rare firsthand perspective on the cutting edge of AI and large language models. In our conversation, we discuss:

• How OpenAI builds product

• What people misunderstand about AI model training

• Differences between how OpenAI and Anthropic operate

• The role of synthetic data in model development

• How to build trust between users and AI models

• Why she moved from engineering to research

• Much more

—

Brought to you by:

• Enterpret—Transform customer feedback into product growth

• Vanta—Automate compliance. Simplify security

• Loom—The easiest screen recorder you’ll ever use

—

Find the transcript at: https://www.lennysnewsletter.com/p/why-soft-skills-are-the-future-of-work-karina-nguyen

—

Where to find Karina Nguyen:

• X: https://x.com/karinanguyen_

• LinkedIn: https://www.linkedin.com/in/karinanguyen28

• Website: https://karinanguyen.com/

—

Where to find Lenny:

• Newsletter: https://www.lennysnewsletter.com

• X: https://twitter.com/lennysan

• LinkedIn: https://www.linkedin.com/in/lennyrachitsky/

—

In this episode, we cover:

(00:00) Introduction to Karina Nguyen

(04:42) Challenges in model training

(08:21) Synthetic data and its importance

(12:38) Creating Canvas

(18:33) Day-to-day operations at OpenAI

(20:28) Writing evaluations

(23:22) Prototyping and product development

(26:57) Building Canvas and Tasks

(33:34) Understanding the job of a researcher

(35:36) The future of AI and its impact on work and education

(42:15) Soft skills in the age of AI

(47:50) AI’s role in creativity and strategy development

(53:34) Comparing Anthropic and OpenAI

(57:11) Innovations and future visions

(01:07:13) The potential of AI agents

(01:11:36) Final thoughts and career advice

—

Referenced:

• What’s in your stack: The state of tech tools in 2025: https://www.lennysnewsletter.com/p/whats-in-your-stack-the-state-of

• Anthropic: https://www.anthropic.com/

• OpenAI: https://openai.com/

• What is synthetic data—and how can it help you competitively?: https://mitsloan.mit.edu/ideas-made-to-matter/what-synthetic-data-and-how-can-it-help-you-competitively

• GPQA: https://datatunnel.io/glossary/gpqa/

• Canvas: https://openai.com/index/introducing-canvas/

• Barret Zoph on LinkedIn: https://www.linkedin.com/in/barret-zoph-65990543/

• Mira Murati on LinkedIn: https://www.linkedin.com/in/mira-murati-4b39a066/

• JSON Schema: https://json-schema.org/

• Anthropic—100K Context Windows: https://www.anthropic.com/news/100k-context-windows

• Claude 3 Haiku: https://www.anthropic.com/news/claude-3-haiku

• A.I. Chatbots Defeated Doctors at Diagnosing Illness: https://www.nytimes.com/2024/11/17/health/chatgpt-ai-doctors-diagnosis.html

• Cursor: https://www.cursor.com/

• How AI will impact product management: https://www.lennysnewsletter.com/p/how-ai-will-impact-product-management

• Lee Byron on LinkedIn: https://www.linkedin.com/in/lee-byron/

• GraphQL: https://graphql.org/

• Claude in Slack: https://www.anthropic.com/claude-in-slack

• Sam Altman on X: https://x.com/sama

• Jakub Pachocki on LinkedIn: https://www.linkedin.com/in/jakub-pachocki/

• Lennybot: https://www.lennybot.com/

• ElevenLabs: https://elevenlabs.io/

• Westworld on Prime Video: https://www.amazon.com/Westworld-Season-1/dp/B01N05UD06

• A conversation with OpenAI’s CPO Kevin Weil, Anthropic’s CPO Mike Krieger, and Sarah Guo: https://www.youtube.com/watch?v=IxkvVZua28k

• Tuple: https://tuple.app/

• How Shopify builds a high-intensity culture | Farhan Thawar (VP and Head of Eng): https://www.lennysnewsletter.com/p/how-shopify-builds-a-high-intensity-culture-farhan-thawar

—

Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email podcast@lennyrachitsky.com.

—

Lenny may be an investor in the companies discussed.

This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.lennysnewsletter.com/subscribe