RoboPapers Ep#20 VideoMimic
Jul 13, 2025
Arthur Allshire and Hongsuk Choi, both PhD students at UC Berkeley, dive into their groundbreaking project, VideoMimic. They discuss how humanoid robots can learn locomotion and interaction through human imitation. Key insights include advancements in 3D reconstruction from videos, the challenges of kinematic retargeting, and the integration of depth mapping technologies. They also touch on the complexity of training robots in diverse environments and the exciting potential of using minimal video data for effective robotic training.
AI Snips
Chapters
Transcript
Episode notes
Video Imitation for Humanoids
- Learning humanoid control via video imitation bypasses manual reward engineering.
- Robots can learn terrain traversal and interaction by distilling human video demonstrations into policies.
Monocular Video to 3D Reconstruction
- Recent advances allow converting monocular video into dense 3D point clouds over time.
- This enables reconstructing the human motion and environment jointly from ordinary videos.
Automatic Scene Scaling
- Scale the scene mesh down to the robot size to ensure physically feasible motions.
- Use automatic scene scaling to reduce embodiment gaps and aid imitation learning.