Bringing a Blurry Frame Alive at High Frame-Rate with an Event Camera

Event: CVPR 2019 Workshop on Event Cameras · Duration: 10 min · ▶ Watch on YouTube

Abstract

This presentation introduces a novel approach to reconstruct high frame-rate and sharp video from a single blurry image and event camera data. The method leverages the complementary nature of event cameras, which provide high temporal resolution event streams, and traditional frame-based cameras, which capture intensity images. By formulating the blur as a double integral model that combines the initial intensity image and the event stream, the system can estimate the latent sharp images. The core challenge lies in determining the contrast threshold parameter, which is addressed by optimizing the cross-correlation between event-integrated edges and reconstructed image edges.

Speakers

  • Yuchao Dai — Northwestern Polytechnical University (presenting for Liyuan Pan)
  • Liyuan Pan — Australian National University
  • Cedric Scheerlinck — Australian Centre for Robotic Vision
  • Xin Yu — Australian National University
  • Richard Hartley — Australian National University
  • Miaomiao Liu — Australian National University

Talks (1)

  • 00:00:00 — Yuchao Dai: Bringing a Blurry Frame Alive at High Frame-Rate with an Event Camera
    • A method to reconstruct high frame-rate, sharp video from a single blurry frame and its associated event data by modeling the blur as a double integral of event-driven intensity changes.

Key Takeaways

  • Event cameras offer high temporal resolution (low latency) data, complementing traditional cameras which provide high spatial resolution but suffer from motion blur in dynamic scenes.
  • The proposed method models blur as a double integral, combining a single blurry frame with event data to reconstruct a high frame-rate, sharp video.
  • A key aspect of the model is the estimation of a contrast threshold parameter ‘c’, which is crucial for accurate reconstruction.
  • The contrast threshold ‘c’ is found by maximizing the cross-correlation between edges derived from event data and edges from the reconstructed image, offering a principled optimization approach.
  • The solution is simple, computationally efficient, and demonstrates improved performance (higher PSNR and SSIM) compared to existing methods that use only events, only images, or combined event-image approaches.

Methods / Models / Datasets Mentioned

  • Event-based Double Integral (EDI) model
  • Fibonacci search

Topics

Event cameras · High frame-rate video · Deblurring · Motion blur · Image reconstruction · Event-based vision · Double integral model · Contrast threshold estimation

Notes

Open for commentary — connections to other work, critiques, follow-up reading.