Architectures: Transformers and Beyond

Fei-Fei Li and Justin Johnson are cofounders of World Labs, who have recently launched Marble (https://marble.worldlabs.ai/), a new kind of generative “world model” that can create editable 3D environments from text, images, and other spatial inputs. Marble lets creators generate persistent 3D worlds, precisely control cameras, and interactively edit scenes, making it a powerful tool for games, film, VR, robotics simulation, and more. In this episode, Fei-Fei and Justin share how their journey from ImageNet and Stanford research led to World Labs, why spatial intelligence is the next frontier after LLMs, and how world models could change how machines see, understand, and build in 3D.

We discuss:

• The massive compute scaling from AlexNet to today and why world models and spatial data are the most compelling way to “soak up” modern GPU clusters compared to language alone.

• What Marble actually is: a generative model of 3D worlds that turns text and images into editable scenes using Gaussian splats, supports precise camera control and recording, and runs interactively on phones, laptops, and VR headsets.

• Fei-fei’s essay (https://drfeifei.substack.com/p/from-words-to-worlds-spatial-intelligence) on spatial intelligence as a distinct form of intelligence from language: from picking up a mug to inferring the 3D structure of DNA, and why language is a lossy, low-bandwidth channel for describing the rich 3D/4D world we live in.

• Whether current models “understand” physics or just fit patterns: the gap between predicting orbits and discovering F=ma, and how attaching physical properties to splats and distilling physics engines into neural networks could lead to genuine causal reasoning.

• The changing role of academia in AI, why Fei-Fei worries more about under-resourced universities than “open vs closed,” and how initiatives like national AI compute clouds and open benchmarks can rebalance the ecosystem.

• Why transformers are fundamentally set models, not sequence models, and how that perspective opens up new architectures for world models, especially as hardware shifts from single GPUs to massive distributed clusters.

• Real use cases for Marble today: previsualization and VFX, game environments, virtual production, interior and architectural design (including kitchen remodels), and generating synthetic simulation worlds for training embodied agents and robots.

• How spatial intelligence and language intelligence will work together in multimodal systems, and why the goal isn’t to throw away LLMs but to complement them with rich, embodied models of the world.

• Fei-Fei and Justin’s long-term vision for spatial intelligence: from creative tools for artists and game devs to broader applications in science, medicine, and real-world decision-making.

—

Fei-Fei Li

• X: https://x.com/drfeifei

• LinkedIn: https://www.linkedin.com/in/fei-fei-li-4541247

Justin Johnson

• X: https://x.com/jcjohnss

• LinkedIn: https://www.linkedin.com/in/justin-johnson-41b43664

Where to find Latent Space

• X: https://x.com/latentspacepod

• Substack: https://www.latent.space/

Chapters

• 00:00:00 Introduction and the Fei-Fei Li & Justin Johnson Partnership
• 00:02:00 From ImageNet to World Models: The Evolution of Computer Vision
• 00:12:42 Dense Captioning and Early Vision-Language Work
• 00:19:57 Spatial Intelligence: Beyond Language Models
• 00:28:46 Introducing Marble: World Labs' First Spatial Intelligence Model
• 00:33:21 Gaussian Splats and the Technical Architecture of Marble
• 00:22:10 Physics, Dynamics, and the Future of World Models
• 00:41:09 Multimodality and the Interplay of Language and Space
• 00:37:37 Use Cases: From Creative Industries to Robotics and Embodied AI
• 00:56:58 Hiring, Research Directions, and the Future of World Labs