Gradient Dissent: Conversations on AI Why Physical AI Needed a Completely New Data Stack
82 snips
Dec 16, 2025 Nikolaus West, CEO and co-founder of Rerun, specializes in revolutionizing how we log and visualize multimodal sensor data for robotics. He discusses the challenges of taking AI from the lab to the real world and how Rerun's innovative data stack solves these issues. Key topics include breakthrough techniques in manipulation, the intriguing blend of reinforcement and imitation learning, and the significance of open-source tools in advancing robotics. Nikolaus also shares insights on the future of consumer robotics and the importance of robust data pipelines.
AI Snips
Chapters
Transcript
Episode notes
Multimodal Time-Aware Data Is The Core
- Rerun logs multimodal, time-varying sensor data and visualizes it to make debugging far easier.
- The stack focuses on data capture, querying, and visualization rather than model training itself.
Prioritize Low-Friction, Composable Logging
- Make logging low friction and flexible so researchers can dump data without thinking.
- Use a composable entity-component style model so visualizers can interpret arbitrary combinations of components.
Manipulation Is Suddenly Practical
- Learning-based manipulation has advanced quickly and tasks like folding laundry moved from impossible to mundane.
- Progress comes from end-to-end learning techniques such as imitation learning and combined RL approaches.