Applications, Software and Hardware for Event-Based Vision

Event: CVPR 2019 Workshop on Event-Based Vision · Duration: 14 min · ▶ Watch on YouTube

Abstract

The speaker introduces event-based vision, highlighting its potential to revolutionize vision systems. He discusses the evolution of DVS technology from early lab prototypes to widespread adoption across various industries. The talk covers software development tools like jAER and the new DV framework, emphasizing ease of use and broad interfacing capabilities. He then delves into application selection, proposing a quantitative framework to determine where DVS offers the most significant benefits over traditional frame-based systems, particularly in scenarios requiring low latency, high dynamic range, or energy efficiency. Finally, he presents hardware innovations, including in-array noise filtering and neuromorphic processors like Dynap-SE and DynapCNN, culminating in the announcement of SPECK, a single-chip DVS+CNN processor for micropower intelligent scene analysis in mobile and IoT applications.

Speakers

  • Kynan Eng — iniVation

Talks (1)

  • 00:00:00 — Kynan Eng: Applications, Software and Hardware for Event-Based Vision
    • This talk covers the evolution of event-based vision, focusing on software development tools like jAER and DV, a quantitative framework for selecting DVS applications, and hardware innovations including in-array noise filtering and integrated DVS+CNN processors for low-power mobile and IoT uses.

Key Takeaways

  • Event-based vision has evolved significantly, with widespread adoption across diverse industries since the DVS128 in 2008.
  • Open-source software frameworks like jAER and the new DV (Dynamic Vision) are crucial for fostering community development and making DVS technology accessible and easier to use.
  • A quantitative framework can help identify optimal applications for DVS based on power budget, frame rate, and energy efficiency, determining when DVS offers superior performance over traditional frame-based systems.
  • Hardware innovations such as in-array noise filtering and integrated DVS+CNN processors (like SPECK) are key to maximizing DVS benefits for ultra-low-power, low-latency applications in mobile and IoT.
  • Effective neuromorphic hardware design adheres to principles of minimal power budget ASICs and activity-dependent computation, avoiding unnecessary processing.

Methods / Models / Datasets Mentioned

  • DVS (Dynamic Vision Sensor)
  • jAER
  • libcaer
  • OpenCV
  • TensorFlow
  • Dynap-SE
  • DynapCNN
  • SPECK

Topics

Event-based vision · DVS · Neuromorphic computing · Software development · Hardware development · Applications · Energy efficiency · Low latency · High Dynamic Range (HDR) · Noise filtering · CNN processor · IoT · Mobile

Notes

Open for commentary — connections to other work, critiques, follow-up reading.