Scaling Video Generation Models for Simulating the Physical World

OpenAI's Sora video model aims at building general-purpose simulators of the physical world by scaling video generation models. The research paper emphasizes the importance of simulating the physical world as a key capacity in their goal of achieving AGI. Sora turns videos into patches using a compression network and generates space-time patches as transformer tokens. By scaling diffusion transformers, Sora produces high-quality video samples, and its sample quality improves with increased training compute.