EVENTNET: ASYNCHRONOUS RECURSIVE EVENT PROCESSING

Event: CVPR 2019 · Duration: 8 min · ▶ Watch on YouTube

Abstract

This paper introduces EventNet, a novel neural network designed for real-time processing of asynchronous event streams from event-based cameras. Unlike traditional frame-based methods or batch-oriented deep learning models like PointNet, EventNet processes events recursively and event-wise, avoiding densification and re-computation. Key innovations include a recursive temporal coding function, a Look-Up Table (LUT) realization for the MLP component for significant speedup, and an asynchronous two-module architecture to efficiently handle variable event rates and on-demand application output. Experimental results demonstrate that EventNet achieves comparable performance to existing methods while being orders of magnitude faster, enabling real-time processing at 1 microsecond per event.

Speakers

  • Yusuke Sekikawa — Denso IT Laboratory Inc., Japan

Talks (15)

  • 00:00:00 — Yusuke Sekikawa: EVENTNET: ASYNCHRONOUS RECURSIVE EVENT PROCESSING
    • Introduction to EventNet, an event-based neural network for asynchronous recursive event processing.
  • 00:00:09 — Yusuke Sekikawa: Motivation: Event-based Processing
    • Discusses the advantages of event-based cameras (HDR, fast, non-redundant data) over frame-based processing and the limitations of existing methods that convert sparse events to dense frames.
  • 00:00:47 — Yusuke Sekikawa: Problem Statement
    • Formalizes the problem of processing asynchronous sparse event data, highlighting requirements for E2E trainability, sparse event-wise processing, real-time recursive processing, and local permutation invariance.
  • 00:01:55 — Yusuke Sekikawa: Related work: Deep Learning on Unordered Sets
    • Introduces PointNet as inspiration for processing unordered sets (like point clouds) directly using symmetric functions to achieve permutation invariance.
  • 00:02:31 — Yusuke Sekikawa: Toward Real-time Processing
    • Explains why PointNet’s batch processing approach is infeasible for real-time event data due to the need to reprocess all events with changing elapsed times for each new event.
  • 00:03:58 — Yusuke Sekikawa: EventNet: Key Components for Real-time
    • Outlines the three main technical contributions of EventNet: Recursive Event-wise processing, LUT Realization of MLP h, and Asynchronously Two Module Architecture.
  • 00:04:26 — Yusuke Sekikawa: Recursive Event-wise processing
    • Details how EventNet achieves recursive computation for temporal coding (c) and max pooling, avoiding re-computation of MLP h for previously processed events.
  • 00:05:05 — Yusuke Sekikawa: Asynchronous recursive event-wise processing
    • Presents the EventNet model’s recursive processing, which is significantly faster than batch processing by computing c and max recursively.
  • 00:05:33 — Yusuke Sekikawa: Temporal coding/max in EventNet
    • Explains the temporal coding function ‘c’ that decays linearly with elapsed time, ensuring that only the newest event can be the maximum and older events ‘die’ out.
  • 00:05:44 — Yusuke Sekikawa: LUT Realization of MLP h
    • Describes how the discrete nature of event camera inputs (x, y, p) allows for pre-computation of the MLP ‘h’ into a Look-Up Table (LUT), resulting in a 45x speedup for inference.
  • 00:06:22 — Yusuke Sekikawa: Asynchronously Two Module Architecture
    • Introduces a two-module architecture where the first module updates global features at the variable event rate, and the second computes output on demand at a lower, uniform application rate.
  • 00:07:02 — Yusuke Sekikawa: Asymmetric Training/Inference
    • Explains that EventNet uses a batch structure for training (to leverage batch normalization and parallelization) and a recursive, asynchronous structure for inference.
  • 00:07:23 — Yusuke Sekikawa: Experiments
    • Details the datasets used for evaluation: ETH-TED+ for object motion estimation and event-wise semantic segmentation, and MVSEC for ego-motion (yaw-rate) estimation.
  • 00:07:31 — Yusuke Sekikawa: Results
    • Presents experimental results showing EventNet achieves comparable performance to PointNet while being ~45x faster, enabling real-time processing at 1 microsecond per event.
  • 00:07:50 — Yusuke Sekikawa: EventNet: Contribution Summary
    • Summarizes the key contributions of EventNet for real-time, asynchronous event stream processing.

Key Takeaways

  • EventNet enables real-time processing of asynchronous event streams by processing each event recursively and event-wise, avoiding the computational overhead of batch processing.
  • The use of a Look-Up Table (LUT) for the MLP component significantly accelerates inference by leveraging the discrete nature of event camera inputs.
  • An asynchronous two-module architecture allows EventNet to handle variable event rates efficiently while providing outputs at a lower, application-driven rate.
  • EventNet achieves comparable accuracy to existing methods like PointNet but with a substantial speedup (approx. 45x faster), demonstrating its suitability for high-speed, low-latency applications.

Methods / Models / Datasets Mentioned

  • EventNet
  • PointNet
  • MLP
  • ETH-TED+
  • MVSEC

Topics

Event-based cameras · Asynchronous event processing · Real-time processing · Recursive neural networks · Permutation invariance · Sparse data processing · Temporal coding · Look-Up Table (LUT) · Asynchronous architecture · Object motion estimation

Notes

Open for commentary — connections to other work, critiques, follow-up reading.