The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence) Imagine while Reasoning in Space: Multimodal Visualization-of-Thought with Chengzu Li - #722
34 snips
Mar 10, 2025 
Chengzu Li, a PhD student at the University of Cambridge, unpacks his pioneering work on Multimodal Visualization-of-Thought (MVoT). He explores the intersection of spatial reasoning and cognitive science, linking concepts like dual coding theory to AI. The conversation includes insights on token discrepancy loss to enhance visual and language integration, challenges in spatial problem-solving, and real-world applications in robotics and architecture. Chengzu also shares lessons learned from experiments that could redefine how machines navigate and reason about their environment.
AI Snips
Chapters
Transcript
Episode notes
Robot Navigation Analogy
- Chengzu Li describes a navigation robot's thought process when asked to get a drink.
- It must locate the refrigerator (kitchen) and plan a route, mirroring MVoT's spatial reasoning.
MVoT Origins
- MVoT arose from the dynamic spatial reasoning task in Top View RS, focusing on navigation paths.
- Inspired by VoLT, it uses visualizations, but with actual images instead of ASCII art.
Dual Coding Theory
- MVoT's design connects to the dual coding theory in cognitive science.
- This theory suggests humans process information through verbal and nonverbal (imagery) channels.