Asynchronous Convolutions and Image Reconstruction

Event: Second International Workshop on Event-based Vision and Smart Cameras 2019 · Duration: 23 min · ▶ Watch on YouTube

Abstract

Robert Mahony presents a systems and control theory framework for processing asynchronous event camera data, emphasizing embedded implementation on FPGA-based platforms like ULTRA96. The approach models event streams as continuous-time signals, enabling the design of asynchronous event filters for tasks such as image reconstruction, bias correction, and gradient estimation. By leveraging linear system theory and explicit solutions to ordinary differential equations, the method achieves computational efficiency and allows for the fusion of low-frequency information from conventional frames with high-frequency event data, leading to robust and real-time vision processing for robotics.

Speakers

  • Robert Mahony — Australian National University, Australian Centre for Robotic Vision

Talks (1)

  • 00:00:00 — Robert Mahony: Asynchronous Convolutions and Image Reconstruction
    • This talk introduces a systems and control theory approach to processing asynchronous event camera data, focusing on embedded implementation using FPGAs for efficient image reconstruction, bias correction, gradient computation, optical flow estimation, and feature tracking.

Key Takeaways

  • Event camera data can be effectively modeled and processed using continuous-time systems and control theory, even with discrete events.
  • FPGA-based embedded systems are crucial for real-time, computationally efficient processing of high-rate asynchronous event streams.
  • Asynchronous event filters, particularly complementary filters, allow for robust image reconstruction and feature extraction by fusing low-frequency conventional image data with high-frequency event data.
  • This framework enables advanced vision tasks like optical flow and feature tracking directly from event streams, with potential for real-time adaptation and bias correction.

Methods / Models / Datasets Mentioned

  • DAVIS240C
  • ULTRA96
  • Kroneker delta function
  • Dirac delta function
  • Sobel kernel
  • Harris Corner Detector

Topics

Asynchronous event cameras · Event stream processing · Embedded systems · FPGA implementation · Image reconstruction · High-pass filtering · Complementary filtering · Optical flow · Feature tracking · Bias correction

Notes

Open for commentary — connections to other work, critiques, follow-up reading.