Miniaturized embedded event based vision for high-speed robots

Event: Conference Presentation · Duration: 28 min · ▶ Watch on YouTube

Abstract

Jörg Conradt from Technische Universität München presents his lab’s work on miniaturized embedded event-based vision systems (eDVS) for high-speed robots. The core idea is to move away from traditional computer vision approaches that reconstruct images, instead processing individual events directly on embedded microcontrollers. This allows for ultra-low latency, low power consumption, and sparse data streams, making these systems ideal for real-time robotic applications. The presentation highlights various projects, from simple robot chain following and high-speed pole balancing to complex SLAM systems and micro-quadrocopters, and even a healthcare device for the visually impaired, demonstrating the versatility and efficiency of event-based vision when integrated directly into hardware.

Speakers

  • Jörg Conradt — Technische Universität München, Neuroscientific System Theory, Competence Center NeuroEngineering

Talks (1)

  • 00:00:00 — Jörg Conradt: Miniaturized embedded event based vision for high-speed robots
    • This presentation introduces miniaturized embedded event-based vision systems (eDVS) developed at TUM, emphasizing their low latency, low power consumption, and direct integration of processing on the sensor board, showcasing various robotic applications from mobile robot chains and pole balancing to SLAM and micro-quadrocopters, and even a healthcare device for the visually impaired.

Key Takeaways

  • Event-based vision offers significant advantages (sparse data, ultra-low latency, high dynamic range) over traditional frame-based cameras for high-speed and embedded robotic applications.
  • Integrating processing directly onto the sensor board (eDVS) allows for real-time algorithm execution with minimal latency and power consumption, overcoming limitations of external computing.
  • Algorithms designed to process individual events rather than reconstructed images are crucial for fully leveraging the benefits of event-based sensors in resource-constrained environments.
  • The presented eDVS systems enable a wide range of applications, from basic tracking and balancing to complex SLAM and autonomous flight in miniaturized robots, and even assistive devices for the visually impaired.
  • The field is progressing towards even smaller, more integrated, and power-efficient neuromorphic systems, pushing the boundaries of autonomous real-time processing.

Methods / Models / Datasets Mentioned

  • DVS128 sensor
  • DAVIS240 sensor
  • DAVIS640 sensor
  • Embedded DVS Systems (eDVS)
  • Microcontroller LPC4337 32bit ARM Cortex
  • MPU-9150 Nine-Axis IMU
  • Continuous Hough-Space
  • Event-based Particle Filter Framework
  • SpiNNaker Real-Time Hardware Interface
  • NST SpinninBot
  • NEURO-GLASSES
  • AuviNav
  • Robo Bee Project

Topics

Event-based vision · Embedded systems · High-speed robotics · Neuromorphic computing · Low-latency vision · SLAM (Simultaneous Localization and Mapping) · Micro-quadrocopters · Healthcare applications · Real-time processing

Notes

Open for commentary — connections to other work, critiques, follow-up reading.