The development of the DVS and DAVIS sensors

Event: CVPR 2025 Workshop on X · Duration: 31 min · ▶ Watch on YouTube

Abstract

Tobi Delbruck presents the development of Dynamic Vision Sensors (DVS) and Dynamic and Active Pixel Vision Sensors (DAVIS), drawing inspiration from the human eye’s event-driven processing. He details the DVS pixel architecture, which outputs asynchronous brightness change events with a wide dynamic range, and the DAVIS pixel, which integrates DVS with a conventional active pixel sensor for intensity readout. Live demonstrations illustrate the sensors’ ability to capture motion with low latency and sparse data output, making them suitable for real-time applications. The talk concludes by exploring the use of DVS data to drive convolutional neural networks for tasks like gesture recognition, emphasizing the advantages of event-based data for efficient deep inference.

Speakers

  • Tobi Delbruck — Inst. of Neuroinformatics, University of Zurich and ETH Zurich

Talks (7)

  • 00:00:00 — Tobi Delbruck: The development of the DVS and DAVIS sensors
    • Introduction to the DVS and DAVIS sensors, their biological inspiration from the human eye, and a comparison with conventional frame-based cameras.
  • 00:05:00 — Tobi Delbruck: DVS (Dynamic Vision Sensor) Pixel
    • Detailed explanation of the DVS pixel architecture, its event-driven output, and its wide dynamic range capabilities demonstrated with a high-contrast scene.
  • 00:08:47 — Tobi Delbruck: DAVIS (Dynamic and Active Pixel Vision Sensor) Pixel
    • Introduction to the DAVIS pixel, which combines the event-based DVS with a conventional active pixel sensor for simultaneous event and intensity data, showcasing its chip architecture and live demos.
  • 00:18:45 — Tobi Delbruck: DVS sensor specifications and event threshold matching measurement
    • Discussion of DVS sensor specifications including power consumption, dynamic range, response latency, and fixed pattern noise (FPN) matching, along with an experimental method for measuring event threshold matching.
  • 00:23:01 — Tobi Delbruck: DVS pixel size trend and commercial developments
    • Analysis of the trend in DVS pixel size reduction over time, comparing it with state-of-the-art image sensors, and an overview of commercial entities and research groups developing event-based cameras.
  • 00:26:02 — Tobi Delbruck: Driving Convolutional Neural Networks with DVS
    • Exploration of using DVS data to drive convolutional neural networks (CNNs) for tasks like gesture recognition (RoShambo), highlighting the benefits of sparse, event-driven input for efficient deep inference.
  • 00:30:09 — Tobi Delbruck: Conclusions
    • Summary of the DVS/DAVIS sensor development, their applications, challenges in pixel size reduction and algorithm development, and the potential for event sensors in driving efficient deep inference.

Key Takeaways

  • DVS and DAVIS sensors emulate key properties of biological retinas, offering wide dynamic range and sparse, asynchronous event-based output.
  • These sensors are particularly useful for real-time applications in uncontrolled conditions due to their low latency and efficient data representation.
  • Event-based data can effectively drive deep inference in CNNs, especially when leveraging sparsity-aware accelerators, opening new avenues for efficient AI processing.
  • While pixel size reduction is a challenge for industry, academia plays a crucial role in developing effective algorithms for these novel sensors.
  • The combination of event-based vision with IMU data allows for robust motion compensation and extraction of relative optical flow.

Methods / Models / Datasets Mentioned

  • DVS
  • DAVIS
  • APS
  • AER
  • CNN
  • LeNet
  • ReLU
  • MaxPool
  • FC layer
  • Caffe
  • NullHop
  • RoShambo
  • IMU
  • Bayesian modeling

Topics

Dynamic Vision Sensors (DVS) · DAVIS sensors · neuromorphic engineering · event-based vision · silicon retina · wide dynamic range · sparse data · asynchronous processing · low latency · power efficiency · convolutional neural networks (CNNs) · deep inference · RoShambo · pixel size trend · sensor specifications

Notes

Open for commentary — connections to other work, critiques, follow-up reading.