Spike timing-based unsupervised learning of orientation, disparity, and motion representations in a spiking neural network

Event: CVPR 2025 · Duration: 3 min · ▶ Watch on YouTube

Abstract

This video presents a spiking neural network (SNN) designed for unsupervised learning of visual representations, specifically orientation, disparity, and motion. Inspired by the human visual cortex, the SNN incorporates two layers (simple and complex cells) and various homeostatic mechanisms like refractory periods and lateral inhibition to regulate its activity. The network learns through a modified Spike Timing-Dependent Plasticity (STDP) rule, demonstrating the ability to develop Gabor-like receptive fields that are sensitive to moving bars of varying speeds and disparities. Furthermore, the complex cells act as a pooling layer, becoming selective to a wide range of directions, and the network successfully estimates depth from stereoscopic event-based camera data in urban environments. Future work aims to integrate this SNN into an active framework for self-calibrating stereo vision with motor control.

Speakers

  • Céline Teulière — Institut Pascal
  • Jochen Triesch — FIAS Frankfurt Institute for Advanced Studies

Talks (1)

  • 00:00:00 — Thomas Barbier: Spike timing-based unsupervised learning of orientation, disparity, and motion representations in a spiking neural network
    • This presentation introduces a spiking neural network that learns visual representations for orientation, disparity, and motion in an unsupervised manner, leveraging spike timing-dependent plasticity and homeostatic mechanisms.

Key Takeaways

  • The presented SNN successfully learns efficient visual representations for orientation, motion, and disparity in an unsupervised manner using event-based camera data.
  • The network’s architecture and learning rules are biologically inspired, incorporating simple and complex cell layers, homeostatic mechanisms, and a modified STDP rule.
  • The SNN demonstrates the ability to learn Gabor-like receptive fields and complex cell responses selective to various directions, and can perform depth estimation comparable to frame-based methods.
  • Future research aims to extend this work into an active framework for self-calibrating stereo vision, integrating reinforcement learning for motor control.

Methods / Models / Datasets Mentioned

  • Leaky Integrate and Fire (LIF) Neuron Model
  • Spike Timing-Dependent Plasticity (STDP)
  • Active Efficient Coding (AEC) framework

Topics

Spiking Neural Networks (SNN) · Unsupervised Learning · Spike Timing-Dependent Plasticity (STDP) · Visual Representation Learning · Orientation Detection · Motion Detection · Disparity Estimation · Event-based Cameras · Homeostatic Mechanisms · Stereo Vision

Notes

Open for commentary — connections to other work, critiques, follow-up reading.