TED Tech The next grand challenge for AI | Jim Fan
4 snips
Mar 29, 2024 
Researcher Jim Fan discusses the next grand challenge in AI: creating a 'foundation agent' to operate in virtual and physical worlds. He envisions this technology impacting video games, metaverses, drones, and humanoid robots. The podcast explores the potential of a single model mastering skills across different realities, including AI capabilities in Minecraft and advancements in lifelong learning and multi-body control.
AI Snips
Chapters
Transcript
Episode notes
Moment AlphaGo Sparked a Broader Question
- Jim Fan recalls watching AlphaGo beat Lee Sedol and feeling awe at AI's milestone achievement.
- He notes that AlphaGo could only play Go and could not transfer skills to other games or physical tasks.
Three Axes Define Agent Capability
- Jim Fan frames agent progress across three axes: skills, embodiments, and realities.
- These axes define how versatile an AI must be to reach general capability.
Voyager Learns by Writing and Testing Code
- Fan describes Voyager, an agent in Minecraft that writes JavaScript to act and learns from code execution, agent state, and world state feedback.
- Voyager stores matured skills in a persistent library and self-improves through reflection and trials.