Event-based Vision for Augmented Reality

Event: CVPR 2019 Workshop on Event-based Vision · Duration: 20 min · ▶ Watch on YouTube

Abstract

This presentation explores the critical role of event-based vision in achieving truly immersive and interactive augmented reality (AR) experiences. The speaker highlights the stringent latency requirements (sub-20ms) for AR devices to trick the human brain, advocating for fast, low-power, and high-temporal-resolution sensing. The talk introduces Insightness’s DAVIS sensor technology, which combines event-based and global shutter frame-based readouts from the same photodiodes, offering complementary strengths for various AR processing tasks. Finally, the presentation unveils the new Rino 4 sensor, an HD-resolution event-based camera designed for compact AR modules.

Speakers

  • Stefan Ialer — Insightness

Talks (1)

  • 00:00:00 — Stefan Ialer: Event-based Vision for Augmented Reality
    • The speaker introduces the concept of augmented reality, emphasizing the need for fast, efficient sensors to enable immersive and interactive experiences with low latency, and highlights how event-based sensors address these requirements.

Key Takeaways

  • Achieving truly immersive AR requires photon-to-photon latency under 20ms, necessitating extremely fast and efficient sensing.
  • Event-based sensors offer low latency, high temporal resolution, and low power consumption, making them ideal for dynamic environment understanding, pose estimation, and user interaction in AR.
  • The DAVIS sensor architecture uniquely combines event-based and global shutter frame-based readouts from shared photodiodes, providing both fast motion detection and high-quality texture information without interference.
  • Insightness’s Rino 3 and the newly introduced Rino 4 (HD resolution) event-based cameras are designed to meet these AR sensing challenges, offering configurable resolutions and integration into compact camera modules.
  • Event-based vision solutions have demonstrated effectiveness in applications like optical flow, drone collision avoidance, and visual inertial odometry, even in challenging lighting conditions and with moving objects.

Methods / Models / Datasets Mentioned

  • The DAVIS
  • Insightness Rino 3
  • Insightness Rino 4
  • Optical Flow Approximations (2016)
  • SW: Drones (2016)
  • SW: Visual Inertial Odometry (2017)

Topics

Augmented Reality · Event-based Vision · Sensor Technology · Low Latency · High Temporal Resolution · Power Efficiency · Mixed Reality Devices · Visual Inertial Odometry · Optical Flow

Notes

Open for commentary — connections to other work, critiques, follow-up reading.