Event-Based Computer Vision at Sony AVS

Event: CVPR 2021 · Duration: 22 min · ▶ Watch on YouTube

Abstract

This presentation introduces Sony Advanced Visual Sensing (AVS) and its pioneering work in event-based computer vision. It delves into the fundamental definition of a temporal contrast event, breaking down its components including illumination change, reflectance change, and optical flow. The speaker then demonstrates how these event characteristics are leveraged in various applications such as high-speed 3D depth sensing using active lighting, robust real-time tracking, and efficient depth sensing through sensor fusion with iToF cameras. The talk highlights the advantages of event-based sensors over traditional frame-based cameras, particularly in scenarios involving fast motion and repetitive structures.

Speakers

  • Christian Brändli — CEO, Sony AVS

Talks (1)

  • 00:00:00 — Christian Brändli: Event-Based Computer Vision at Sony AVS
    • An overview of event-based computer vision at Sony AVS, covering the fundamental definition of temporal contrast events, their components, and applications in high-speed 3D, tracking, and sensor fusion.

Key Takeaways

  • Event-based vision offers significant advantages over traditional frame-based cameras for fast motion and dynamic environments due to its high temporal resolution.
  • A temporal contrast event encodes illumination change, reflectance change, and optical flow, which can be leveraged for various computer vision tasks.
  • Event-based sensors enable high-speed 3D depth sensing and robust real-time tracking, outperforming traditional methods like OpenCV Lucas-Kanade in challenging scenarios.
  • Sensor fusion of event-based vision with other modalities like iToF can lead to more efficient and accurate depth sensing solutions by reducing the need for continuous high-power measurements.
  • Understanding the underlying physics and noise characteristics of event-based sensors is crucial for developing effective algorithms and applications, especially when dealing with the interplay of signal and noise in event generation.

Methods / Models / Datasets Mentioned

  • Temporal Contrast Event
  • OCV LK
  • iToF
  • Contrast Maximization

Topics

Event-based vision · Temporal contrast · Image sensors · Computer vision · Depth sensing · 3D reconstruction · Object tracking · Sensor fusion · Active lighting · Optical flow · Virtual reality · Augmented reality · Collision avoidance

Notes

Open for commentary — connections to other work, critiques, follow-up reading.