Learning Event-Based Height From Plane and Parallax

Event: CVPR 2019 Workshop on Event-based Vision · Duration: 3 min · ▶ Watch on YouTube

Abstract

This work presents a method for learning event-based height and depth from plane and parallax. The approach utilizes the EVFlowNet architecture, incorporating a novel loss function derived from Plane and Parallax (P+P) methods. During training, the system requires camera-to-ground calibration, time-synchronized image streams, and ground truth odometry. For inference, only the camera-to-ground calibration is necessary to explicitly extract height and depth, offering flexibility for deployment across different vehicles. The method is evaluated on the MVSEC dataset, demonstrating its ability to predict scene structure even in challenging outdoor night and motorcycle scenarios where ground truth information for obstacles is not always available.

Speakers

  • Kenneth Chaney — Penn Engineering GRASP Laboratory

Talks (1)

  • 00:00:00 — Kenneth Chaney: Learning Event-Based Height From Plane and Parallax
    • This presentation introduces a novel approach for predicting height and depth in a scene using event-based cameras, leveraging the EVFlowNet architecture with a Plane and Parallax-based loss function.

Key Takeaways

  • The proposed method uses a novel loss function based on Plane and Parallax to predict scene height and depth from event-based camera data.
  • It leverages the EVFlowNet architecture for robust event-based optical flow estimation.
  • During training, ground truth odometry and camera-to-ground calibration are used, but only calibration is needed for inference, allowing for flexible deployment.
  • The approach effectively handles parallax effects, which increase with object height and camera movement, by correcting for points on the ground plane.
  • Results on the MVSEC dataset demonstrate the method’s ability to estimate height and depth in various challenging outdoor scenarios, including those without explicit ground truth for obstacles.

Methods / Models / Datasets Mentioned

  • EVFlowNet
  • Plane and Parallax (P+P)
  • ground plane homography
  • residual flow
  • MVSEC dataset

Topics

event-based vision · height estimation · depth estimation · plane and parallax · EVFlowNet · neural networks · ground plane homography · MVSEC dataset

Notes

Open for commentary — connections to other work, critiques, follow-up reading.