Solving Real-World Challenges of Large-Scale AV Deployment

Event: CVPR 2025 Workshop on Autonomous Driving · Duration: 28 min · ▶ Watch on YouTube

Abstract

Waymo has achieved substantial scale in autonomous vehicle deployment, now serving over 250,000 paid rides weekly across multiple cities and expanding rapidly. This presentation delves into the real-world challenges encountered during this large-scale operation, particularly focusing on ‘long-tail’ problems that are crucial for robust AV systems. Key areas discussed include navigating diverse weather conditions (fog, rain, snow, dust storms, floods), handling complex occlusions, and the importance of scene understanding in dynamic environments. The talk also touches upon Waymo’s approach to data curation and the development of foundation models to address these intricate problems, aiming to build the world’s most trusted driver.

Speakers

  • Chen Wu — Waymo

Talks (1)

  • 00:04 — Chen Wu: Solving Real-World Challenges of Large-Scale AV Deployment
    • This talk highlights Waymo’s significant progress in autonomous driving deployment, emphasizing the challenges of scaling operations, handling diverse real-world scenarios like extreme weather and occlusions, and the critical role of data curation and advanced AI models in ensuring safety and efficiency.

Key Takeaways

  • Waymo has achieved significant operational scale, performing over 1 million trips per month, demonstrating the viability of autonomous driving in multiple cities.
  • Addressing ‘long-tail’ problems, such as extreme weather conditions (monsoons, snow, floods, dust storms) and complex occlusion scenarios, is paramount for safe and reliable large-scale AV deployment.
  • The Waymo Driver incorporates extensive hardware redundancy (power, braking, steering, compute) and advanced software capabilities to ensure safety and operational continuity even during transient hardware failures.
  • Waymo leverages a Foundation Model architecture that combines AV-specific ML advances with general world knowledge from VLMs to enhance generalization and adaptability across diverse geographical locations and vehicle platforms.
  • Effective data curation, including natural language search, similarity search, few-shot learning, and active learning, is crucial for efficiently identifying and addressing challenging scenarios to continuously improve the AV system.

Methods / Models / Datasets Mentioned

  • Deep Convolutional Networks
  • Transformers
  • Waymo Open Dataset
  • SOTA Deep Learning
  • VLM (Visual Language Models)
  • LLMs (Large Language Models)
  • Waymo Foundation Model
  • Generative AI Behavior model
  • Generative Sim Model
  • Natural language search
  • Similarity search based on appearance and behavior
  • Few-shot learning
  • Active learning

Topics

Autonomous Driving Deployment · Large-Scale Operations · Real-World Challenges · Long-Tail Problems · Weather Resilience · Occlusion Reasoning · Scene Understanding · Data Curation · Foundation Models · Operational Excellence

Notes

Open for commentary — connections to other work, critiques, follow-up reading.