Sergey Levine and Chelsea Finn from Physical Intelligence discuss a realistic path to robotic foundation models, key factors for the future of robotics, and the transformerification of robotics. They explore the shift towards horizontal robotics companies and the importance of building general robotics models for various tasks.
Robotic learning requires real-world data and evaluation, different from language models.
Reducing the cost of robots through innovations can drive the success of robotic learning.
Deep dives
Challenges in Robotic Learning Progress
Robotic learning faces unique challenges compared to training language or visual models. Unlike language models, robotic systems require real-world data and evaluation. Multi-robot policies and robotic instruction prompting in plain text are emerging trends that could revolutionize autonomous systems. Building a successful robotic foundation model company entails deploying robots in the real world or funding sufficient robotics work for data gathering.
Future Outlook for Robotics Industry
Reducing the cost of robots is essential for the widespread success of robotic learning. Lower costs, possibly achieved through innovations like those seen in battery or solar panel manufacturing, could propel robotics forward. The potential for text-based robotic control and teleoperation markets offers new avenues for robot learning in personal and commercial domains. Collaboration between new robotic foundation model labs and text-to-video models promises advancements in understanding physical reality.
1.
Exploring Progress and Challenges in Robotics with a Focus on Foundation Models
0:00 A realistic path to robotic foundation models 2:51 Key factors for the future of robotics 6:19 Everything is a token: The transformerification of robotics