Development of Event-based Sensor and Applications

Event: CVPR 2021 Workshop on Event-based Vision · Duration: 16 min · ▶ Watch on YouTube

Abstract

This presentation introduces the latest advancements in event-based sensors, specifically the CeleX-Family, developed by Omnivision Sensor Solutions. It delves into the sensor architecture, highlighting features like the Event Packer for MIPI compatibility and the in-pixel time-stamp for improved accuracy. The talk discusses the non-idealities of event detection and readout, proposing solutions like faster readout speeds and sequential scanning. Finally, it showcases practical applications such as a portable event-driven Human-Computer Interaction (HCI) system for gesture control and a hybrid attention assessment system utilizing both event-based and traditional CMOS image sensors.

Speakers

  • Shoushun Chen — Omnivision Sensor Solutions (Shanghai) Co. Ltd

Talks (1)

  • 00:00:00 — Shoushun Chen: Development of Event-based Sensor and Applications
    • An overview of Omnivision’s CeleX-Family event-based sensors, their architecture, time-stamp accuracy considerations, and applications in HCI and attention assessment.

Key Takeaways

  • Omnivision’s CeleX-Family of event-based sensors offers high resolution, low power, and advanced features like dynamic event rate control and power management.
  • In-pixel time-stamping is crucial for achieving high accuracy in event-based sensors, especially in scenarios with high event rates and closely timed motions, mitigating issues from readout delays.
  • The real event model accounts for non-idealities like in-pixel latency and readout delay, which can lead to missed events and unpredictable time-stamp sequences in off-pixel time-stamping approaches.
  • Event-based sensors enable novel applications such as low-latency, low-computational-cost Human-Computer Interaction (HCI) systems for gesture control, achieving high accuracy on ARM Cortex-A8 CPUs.
  • Temporally Ordered Space-time Event Clouds (ST-EVNet) can effectively learn features from temporal local regions for gesture recognition, leveraging the inherent temporal information in event streams.

Methods / Models / Datasets Mentioned

  • CeleX-IV
  • CeleX-V
  • CeleX-VI
  • Event Packer
  • ST-EVNet
  • CMOS Image Sensor (CIS)

Topics

Event-based sensors · CeleX-Family · Sensor architecture · Time-stamp accuracy · In-pixel time-stamp · Readout latency · Human-Computer Interaction (HCI) · Gesture recognition · Attention assessment · Event clouds

Notes

Open for commentary — connections to other work, critiques, follow-up reading.