We Taught AI to Play Games—Now It’s a $3.6 Million Company

212 snips

Oct 15, 2025

Join Alex Duffy, the Head of AI Training and brains behind Good Start Labs, as he discusses the innovative role of games in training AI models. He reflects on lessons learned from games like RuneScape and reveals why traditional testing benchmarks are outdated. By using games like Diplomacy, Alex showcases how AI can develop negotiation skills and strategic thinking. He also shares insights on the future of AI in fields like education and life sciences, emphasizing how gaming can demystify AI for the public.

Ask episode

AI Snips

Chapters

Books

Transcript

Episode notes

00:00 / 00:00

Diplomacy Prototype Drew Viral Attention

Alex and Tyler built an AI version of Diplomacy to study model negotiation and strategy.
The launch drew Twitch viewers and viral attention, proving public interest.

00:00 / 00:00

Static Benchmarks Break Down

Static benchmarks saturate and mislead because models can be taught to 'teach to the test.'
Games like Diplomacy provide dynamic, head-to-head evaluations that reveal real model strengths and weaknesses.

00:00 / 00:00

Models Exhibit Distinct Play Styles

Models show distinct personalities: some scheme, others execute reliably, and cost/performance trade-offs matter.
Evaluations must measure behavior style, honesty, speed, and cost together.

Get the Snipd Podcast app to discover more snips from this episode

Get the app

This episode is a little different from our usual fare: It’s a conversation with our head of AI training Alex Duffy about Good Start Labs, a company he incubated inside Every. Today, Good Start Labs is spinning out of Every as a separate company with $3.6 million in funding from General Catalyst, Inovia, Every, and a group of angel investors from top-tier AI labs like DeepMind. We get into how Alex learned some of his biggest lessons about the real world from games, starting with RuneScape, which taught him how markets work and how not to get scammed. He explains why the static benchmarks we use to evaluate LLMs today are breaking down, and how games like Diplomacy offer a richer, more dynamic way to test and train large language models. Finally, Alex shares where he sees the most promise in AI—software, life sciences, and education—and why he believes games can make the models we use smarter, while helping people understand and use AI more effectively.

If you found this episode interesting, please like, subscribe, comment, and share.

Want even more?

Sign up for Every to unlock our ultimate guide to prompting ChatGPT here: https://every.ck.page/ultimate-guide-to-prompting-chatgpt. It’s usually only for paying subscribers, but you can get it here for free.

To hear more from Dan Shipper:

Subscribe to Every: https://every.to/subscribe
Follow him on X: https://twitter.com/danshipper

Timestamps

00:00:00 - Start

00:01:48 - Introduction

00:04:14 - Why evals and benchmarks are broken

00:07:13 - The sneakiest LLMs in the market

00:13:00 - A competition that turns prompting into a sport

00:15:49 - Building a business around using games to make AI better

00:22:39 - Can language models learn how to be funny

00:25:31 - Why games are a great way to evaluate and train new models

00:26:58 - What child psychology tells us about games and AI

00:30:10 - Using games to unlock continual learning in AI

00:36:42 - Why Alex cares deeply about games

00:44:37 - Where Alex sees the most promise in AI

00:50:54 - Rethinking how young people start their careers in the age of AI

Links to resources mentioned in the episode:

Alex Duffy: alex duffy (@alxai_)
Good Start Labs: https://goodstartlabs.com/, good start (@goodstartlabs)
The book Alex is reading about the importance of games: Playing with Reality: How Games Shape Our World
The book Dan recommends by the psychoanalyst D.W. Winnicott: Playing and Reality