The Perception Data Set and the Modeling Challenge

In episode 69 of The Gradient Podcast, Daniel Bashir speaks to Drago Anguelov.

Drago is currently a Distinguished Scientist and Head of Research at Waymo, where he joined in 2018. Earlier, he spent eight years at Google working on 3D vision and pose estimation for StreetView, then leading a research team that developed computer vision systems for annotating Google Photos. He has been involved in developing popular neural network methods such as the Inception architecture and the SSD detector. Before joining Waymo, he also led the 3D perception team at Zoox.

Have suggestions for future podcast guests (or other feedback)? Let us know here or reach us at editor@thegradient.pub

Subscribe to The Gradient Podcast:  Apple Podcasts  | Spotify | Pocket Casts | RSSFollow The Gradient on Twitter

Outline:

* (00:00) Intro

* (02:04) Drago’s background in AI and self-driving, work with Daphne Koller + Sebastian Thrun, computer vision / pose estimation

* (14:20) One- and two-stage object detectors

* (15:15) Early experiences and thoughts on self-driving and its prospects

* (21:00) An introduction to the “self-driving stack”: mapping & localization, perception, behavior modeling & planning, simulation

* (29:25) From Stuart Russell’s comments on early Waymo’s “old-fashioned” approach

* (37:34) Scaling 3D Detection: challenges and architectural innovations

* (43:20) Behavior modeling: making decisions and modeling interactions in multi-agent environments

* (52:42) Distributional RL (+ imitation learning) in self-driving?

* (54:10) The Waymo Open Dataset

* (1:01:48) Looking forward in self-driving

* (1:04:36) Outro

Links:

* Drago’s LinkedIn and Twitter

* Research

* SSD: Single-Shot Multibox Detector

* SCAPE: Shape completion and animation of people

* Behavior Models for Autonomous Driving

* Wayformer

* Symphony: Learning Realistic and Diverse Agents for Autonomous Driving Simulation

* Imitation Is Not Enough

* Scaling 3D Detection to the Long Tail