Detecting Stable Keypoints from Events through Image Gradient Prediction

Event: CVPR 2020 · Duration: 3 min · ▶ Watch on YouTube

Abstract

This work introduces a novel method for detecting stable keypoints from event camera streams, focusing on high-speed processing and low memory footprint. The core idea is to predict image gradients from events using a shallow recurrent convolutional neural network, leveraging the inherent similarity between event data and image gradients. This approach allows for the detection of stable corner points, which are then tracked over significantly longer durations compared to existing event-based methods, while maintaining comparable reprojection error. The method is designed to reduce data analysis load and enable downstream tasks like SLAM and SfM.

Speakers

  • Philippe Chiberre — Prophesee, Paris, France; LIGM, Ecole des Ponts, Univ Gustave Eiffel, CNRS, Marne-la-Vallée, France
  • Etienne Perot — Prophesee, Paris, France
  • Amos Sironi — Prophesee, Paris, France
  • Vincent Lepetit — LIGM, Ecole des Ponts, Univ Gustave Eiffel, CNRS, Marne-la-Vallée, France

Talks (1)

  • 00:00 — Philippe Chiberre: Detecting Stable Keypoints from Events through Image Gradient Prediction
    • This paper presents a method for detecting and tracking stable keypoints from event camera streams at high speed and low memory footprint by predicting image gradients using a shallow recurrent convolutional neural network.

Key Takeaways

  • A novel method for stable keypoint detection and tracking from event streams is proposed, offering high speed and low memory footprint.
  • The method utilizes a shallow recurrent convolutional neural network to predict image gradients directly from event data.
  • Predicting image gradients is highlighted as a more efficient approach for keypoint detection than reconstructing full images.
  • The approach achieves significantly longer keypoint tracks (5 times longer) compared to state-of-the-art event-based methods, while maintaining similar reprojection error.
  • The detected stable keypoints are suitable for enabling downstream tasks such as Simultaneous Localization and Mapping (SLAM) and Structure from Motion (SfM).

Methods / Models / Datasets Mentioned

  • Harris score
  • ConvRNN
  • SE-ResNet
  • evHarris
  • evFast
  • Arc
  • SILC*
  • ATIS Corner Dataset

Topics

Event cameras · Keypoint detection · Image gradient prediction · Recurrent convolutional neural networks · Corner tracking · SLAM (Simultaneous Localization and Mapping) · SfM (Structure from Motion) · Low memory footprint · High speed processing · Harris corner detector

Notes

Open for commentary — connections to other work, critiques, follow-up reading.