EV-SegNet: Semantic Segmentation for Event-based Cameras

Event: CVPR 2019 Workshop on Event-based Vision · Duration: 2 min · ▶ Watch on YouTube

Abstract

This work introduces EV-SegNet, a method for semantic segmentation using event-based cameras. The core idea is to leverage event data, which captures illumination changes, as input for per-pixel classification. To address the lack of labeled event-based datasets for semantic segmentation, the authors extended the DDD17 dataset by generating semantic segmentation pseudo-labels. This was achieved by training a CNN on grayscale Cityscapes images and then applying this trained model to the DDD17 grayscale images to infer labels. The proposed EV-SegNet architecture is an encoder-decoder CNN that takes a 6-channel event representation (event histogram, mean, and standard deviation of timestamps per polarity) as input.

Speakers

  • Iñigo Alonso — Universidad de Zaragoza
  • Ana C. Murillo — Universidad de Zaragoza

Talks (1)

  • 00:00:00 — Iñigo Alonso: EV-SegNet: Semantic Segmentation for Event-based Cameras
    • A presentation on EV-SegNet, a novel approach for semantic segmentation using event-based cameras, which involves extending the DDD17 dataset with pseudo-labels generated from Cityscapes-trained CNNs and using a 6-channel event representation for an encoder-decoder CNN.

Key Takeaways

  • EV-SegNet is presented as a novel approach for semantic segmentation using event-based cameras.
  • The work addresses the challenge of limited labeled event-based data by extending the DDD17 dataset with pseudo-labels.
  • Pseudo-label generation involves training a CNN on Cityscapes grayscale images and applying it to DDD17 grayscale images.
  • The proposed network uses a 6-channel event representation as input for an encoder-decoder CNN.
  • The code for EV-SegNet is publicly available on GitHub.

Methods / Models / Datasets Mentioned

  • EV-SegNet
  • DDD17 dataset
  • Cityscapes dataset
  • Encoder-decoder CNN

Topics

Semantic Segmentation · Event-based Cameras · Event Data · Deep Learning · Convolutional Neural Networks (CNN) · Dataset Extension · Pseudo-labeling · Illumination Changes · Computer Vision

Notes

Open for commentary — connections to other work, critiques, follow-up reading.