The Most Optimum Strategy Against Humans

The average, the agent here sends and receives an average of 292 messages per game. Each turns actions occur simultaneously after non-binding private negotiations. Success requires building trust with others in an environment that encourages players not to trust anyone. And we're going to see how they train a language model that does it.

Transcript

chevron_right

Play full episode

chevron_right

Transcript

Episode notes

#ai #cicero #diplomacy

A team from Meta AI has developed Cicero, an agent that can play the game Diplomacy, in which players have to communicate via chat messages to coordinate and plan into the future.

Paper Title: Human-level play in the game of Diplomacy by combining language models with strategic reasoning

Commented game by human expert: https://www.youtube.com/watch?v=u5192bvUS7k

OUTLINE:

0:00 - Introduction

9:50 - AI in cooperation games

13:50 - Cicero agent overview

25:00 - A controllable dialogue model

36:50 - Dialogue-conditional strategic planning

49:00 - Message filtering

53:45 - Cicero's play against humans

55:15 - More examples & discussion

Homepage: https://ai.facebook.com/research/cicero/

Code: https://github.com/facebookresearch/diplomacy_cicero

Blog: https://ai.facebook.com/blog/cicero-ai-negotiates-persuades-and-cooperates-with-people/

Paper: https://www.science.org/doi/10.1126/science.ade9097

Abstract:

Despite much progress in training AI systems to imitate human language, building agents that use language to communicate intentionally with humans in interactive environments remains a major challenge. We introduce Cicero, the first AI agent to achieve human-level performance in Diplomacy, a strategy game involving both cooperation and competition that emphasizes natural language negotiation and tactical coordination between seven players. Cicero integrates a language model with planning and reinforcement learning algorithms by inferring players' beliefs and intentions from its conversations and generating dialogue in pursuit of its plans. Across 40 games of an anonymous online Diplomacy league, Cicero achieved more than double the average score of the human players and ranked in the top 10% of participants who played more than one game.

Authors: Anton Bakhtin, Noam Brown, Emily Dinan, Gabriele Farina, Colin Flaherty, Daniel Fried, Andrew Goff, Jonathan Gray, Hengyuan Hu, Athul Paul Jacob, Mojtaba Komeili, Karthik Konath, Minae Kwon, Adam Lerer, Mike Lewis, Alexander H. Miller, Sasha Mitts, Adithya Renduchintala, Stephen Roller, Dirk Rowe, Weiyan Shi, Joe Spisak, Alexander Wei, David Wu, Hugh Zhang, Markus Zijlstra

Links:

Homepage: https://ykilcher.com

Merch: https://ykilcher.com/merch

YouTube: https://www.youtube.com/c/yannickilcher

Twitter: https://twitter.com/ykilcher

Discord: https://ykilcher.com/discord

LinkedIn: https://www.linkedin.com/in/ykilcher

If you want to support me, the best thing to do is to share out the content :)

If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):

SubscribeStar: https://www.subscribestar.com/yannickilcher

Patreon: https://www.patreon.com/yannickilcher

Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq