Waymo Co-CEO Explains Technical Limitations of Camera-Only Self-Driving Systems

The Debate on Sensor Requirements for Full Autonomy

Waymo co-CEO Dmitri Dolgov presented a technical argument regarding the sensor requirements for achieving full self-driving autonomy during a talk at Y Combinator’s Startup School. He addressed the long-standing debate on necessary sensor types, specifically critiquing systems that rely solely on cameras.

Limitations of Camera-Only Systems

Dolgov explained that while camera-only systems can approximate human driving performance and support driver-assist products, they encounter a safety ceiling that prevents them from reaching the 'superhuman' performance required for full autonomy. He stated that 'weak sensing' leads to a safety curve that flattens out prematurely, making it unsuitable for vehicles operating without human intervention.

Waymo's Multi-Sensor Approach

Waymo integrates three primary sensing technologies: cameras, lidar, and radar. Cameras offer high resolution and color but are passive and perform poorly in low light or glare. Lidar actively measures 3D structures, while radar penetrates adverse weather conditions like fog, rain, and snow, and directly measures velocity. Both lidar and radar are active sensors, maintaining performance in darkness or bright sunlight.

Complementary Sensor Fusion

Dolgov emphasized that these different sensing modalities are not merely backups but provide distinct, complementary data. Each sensor type processes its own information, which is then fused into a unified environmental model. This integrated view is described as 'vastly superior' to what any single sensor could provide, enabling the system to handle complex scenarios, such as a dust storm where cameras fail but lidar can still detect pedestrians.