EFI-Net: Video Frame Interpolation from Fusion of Events and Frames

Event: CVPR 2025 · Duration: 3 min · ▶ Watch on YouTube

Abstract

This work presents EFI-Net, a novel approach to video frame interpolation that leverages the fusion of data from both conventional intensity images and event streams. The goal is to synthesize non-existent frames by interpolating over a set of sequential frames, utilizing the high temporal resolution of event cameras for precise temporal frame synthesis. The proposed full CNN solution operates in three phases: fusing intensity and event data, up-scaling the output to conventional camera resolution, and colorizing the intensity image to produce a final color frame. EFI-Net demonstrates superior performance compared to existing methods, especially in scenarios with lower input frame rates.

Speakers

  • Genady Paikin — Samsung Israel R&D Center
  • Yotam Ater — Samsung Israel R&D Center
  • Roy Shaul — Samsung Israel R&D Center
  • Evgeny Soloveichik — Samsung Israel R&D Center

Talks (1)

  • 00:00:00 — Genady Paikin: EFI-Net: Video Frame Interpolation from Fusion of Events and Frames
    • This presentation introduces EFI-Net, a full CNN solution for video frame interpolation that fuses data from intensity images and event streams to synthesize precise temporal frames.

Key Takeaways

  • EFI-Net utilizes event camera data, which offers high temporal resolution, to significantly improve video frame interpolation accuracy and sharpness.
  • The proposed architecture is a full CNN solution structured into three distinct phases: data fusion, spatial up-scaling, and colorization.
  • Events are processed by binning continuous time data into tensor channels, with each event influencing the two nearest bins.
  • EFI-Net consistently outperforms previous works like SSM and DAIN, achieving higher PSNR values on both UZH and custom datasets.
  • The method demonstrates robustness, particularly when dealing with lower input frame rates, making it less vulnerable to motion blur or artifacts.

Methods / Models / Datasets Mentioned

  • EFI-Net
  • SSM
  • DAIN

Topics

Video Frame Interpolation · Event Cameras · Convolutional Neural Networks (CNN) · Data Fusion · Temporal Frame Synthesis · High Temporal Resolution · Intensity Images · Event Stream Processing · Image Colorization · Spatial Resolution Upscaling

Notes

Open for commentary — connections to other work, critiques, follow-up reading.