Object and Action Recognition on the Event-Based IBM TrueNorth Processor

Event: Unknown Conference at Marina Bay Sands 2017 · Duration: 19 min · ▶ Watch on YouTube

Abstract

This presentation introduces the IBM TrueNorth neurosynaptic processor, an event-based architecture designed for energy-efficient computation. Unlike conventional frame-based processors, TrueNorth natively processes sparse, asynchronous event streams, making it ideal for dynamic vision sensor (DVS) cameras. The talk details TrueNorth’s architecture, including its million spiking neurons, billions of synapses, and thousands of identical cores, emphasizing its low power consumption (70mW typical) and high event processing rate. It also covers the Corelet Programming Language, a MATLAB-based tool for developing applications on TrueNorth, and demonstrates its use in gesture recognition and TV remote control applications with DVS128 camera data. The presentation concludes by discussing Eedn (Energy-efficient deep networks for TrueNorth), a method to map convolutional neural networks onto TrueNorth while adhering to its unique constraints, achieving near state-of-the-art accuracy on various benchmarks with significantly reduced power.

Speakers

  • Dharmendra Modha — IBM Research

Talks (1)

  • 00:00:00 — Dharmendra Modha: Object and Action Recognition on the Event-Based IBM TrueNorth Processor
    • A presentation on leveraging the IBM TrueNorth neurosynaptic processor for event-based object and action recognition, highlighting its architecture, programming model, and energy-efficient deep learning capabilities for DVS camera data.

Key Takeaways

  • IBM TrueNorth is a neurosynaptic processor designed for energy-efficient, event-based computation, offering a million spiking neurons and billions of synapses.
  • It excels at processing sparse, asynchronous event streams from DVS cameras, providing a native fit for dynamic vision tasks.
  • The Corelet Programming Language simplifies the development of complex applications on TrueNorth, abstracting hardware details.
  • Energy-efficient deep learning (Eedn) techniques allow mapping CNNs to TrueNorth, achieving high accuracy on benchmarks despite hardware constraints like low precision and non-differentiable neurons.
  • TrueNorth demonstrates practical applications like real-time gesture recognition and TV remote control with DVS cameras, showcasing its potential for low-power, high-performance edge AI.

Methods / Models / Datasets Mentioned

  • IBM TrueNorth
  • iniLabs DVS128
  • Xilinx Zynq Z-7020 SoC
  • Corelet Programming Language
  • MatConvNet
  • Eedn (Energy-efficient deep networks for TrueNorth)
  • Gen2 Samsung DVS

Topics

IBM TrueNorth · Neuromorphic computing · Event-based processing · Dynamic Vision Sensors (DVS) · Gesture recognition · Energy-efficient deep learning · Spiking Neural Networks (SNNs) · Corelet Programming Language · Convolutional Neural Networks (CNNs) · Real-time action recognition

Notes

Open for commentary — connections to other work, critiques, follow-up reading.