Robust sensor fusion against on-vehicle sensor staleness

Event: CVPR 2025 · Duration: 4 min · ▶ Watch on YouTube

Abstract

Sensor fusion is critical for autonomous vehicle perception, but sensor staleness, where data from different sensors arrive with varying delays, poses significant challenges. This presentation introduces a novel approach to mitigate the effects of sensor staleness by incorporating a per-point timestamp offset feature during training and simulating stale sensor data through augmentation. The method helps the model learn to handle temporal and spatial misalignments, leading to robust performance even when faced with stale sensor inputs, without significantly impacting system latency.

Speakers

  • Meng Fan — Zoox Inc
  • Yifan Zuo — Zoox Inc
  • Patrick Blaes — Zoox Inc
  • Harley Montgomery — Zoox Inc
  • Subhasis Das — Zoox Inc

Talks (1)

  • 00:00:00 — Meng Fan: Robust sensor fusion against on-vehicle sensor staleness
    • This talk presents an approach to robust sensor fusion for autonomous vehicles by addressing sensor staleness through per-point timestamp offset features and stale sensor data augmentation, demonstrating improved performance on misaligned data.

Key Takeaways

  • Sensor staleness significantly degrades the performance of baseline sensor fusion models in autonomous vehicles.
  • Incorporating a per-point timestamp offset feature helps models learn to handle staleness and spatial misalignment.
  • Simulating stale sensor data through augmentation during training improves model robustness against real-world sensor staleness.
  • The proposed approach is model-agnostic, easy to integrate, and has negligible impact on system latency, while maintaining performance on perfectly synchronized data.

Methods / Models / Datasets Mentioned

  • PointPillar
  • Transformer-based detection model

Topics

Sensor fusion · Autonomous vehicles · Sensor staleness · Data augmentation · Timestamp offset · LiDAR · Camera · Radar · Perception systems · Robustness

Notes

Open for commentary — connections to other work, critiques, follow-up reading.