Real-Time 6DOF Pose Relocalization for Event Cameras with Stacked Spatial LSTM Networks

Event: Unknown Conference 2019 · Duration: 4 min · ▶ Watch on YouTube

Abstract

This presentation introduces a real-time 6DOF pose relocalization method for event cameras, relying solely on event streams. The proposed approach first creates an event image from a list of events within a short time interval. It then employs a stacked spatial LSTM network, comprising a CNN for deep feature extraction and an LSTM for learning spatial dependencies, to predict the camera pose. Experimental results demonstrate that this method significantly outperforms state-of-the-art techniques and generalizes well to unseen event images.

Speakers

  • Anh Nguyen — Istituto Italiano di Tecnologia
  • Thanh-Toan Do — Istituto Italiano di Tecnologia
  • Darwin G. Caldwell — Imperial College London
  • Nikos G. Tsagarakis — Istituto Italiano di Tecnologia

Talks (1)

  • 00:00:00 — Anh Nguyen: Real-Time 6DOF Pose Relocalization for Event Cameras with Stacked Spatial LSTM Networks
    • A presentation on a novel method for real-time 6DOF pose relocalization using event cameras, leveraging a stacked spatial LSTM network for improved accuracy and generalization.

Key Takeaways

  • A novel SP-LSTM architecture is proposed for real-time 6DOF pose relocalization using event cameras.
  • The method utilizes a CNN for feature extraction and an SP-LSTM for learning spatial relationships from event images.
  • The proposed SP-LSTM significantly outperforms existing state-of-the-art methods on the event-camera dataset.
  • The SP-LSTM demonstrates strong generalization capabilities on unseen event images, as shown by novel split experiments.
  • The source code for the method is publicly available on GitHub.

Methods / Models / Datasets Mentioned

  • SP-LSTM Architecture
  • CNN
  • LSTM
  • PoseNet
  • Bayesian PoseNet
  • PoseNet-CNN
  • Event-camera dataset (Mueggler et al. IJRR17)

Topics

6DOF Pose Relocalization · Event Cameras · Stacked Spatial LSTM Networks · Real-Time Systems · Deep Learning · Computer Vision · Feature Extraction · Spatial Dependencies

Notes

Open for commentary — connections to other work, critiques, follow-up reading.