What does Embodied Intelligence mean? Lessons Learned from Drone Racing

Event: CVPR 2025 Workshop on Embodied Intelligence for Autonomous Systems on the Horizon · Duration: 0 min · ▶ Watch on YouTube

Abstract

This talk explores the concept of embodied intelligence through the lens of autonomous drone racing. It highlights the challenges of developing AI systems that can operate in complex, dynamic, and unpredictable real-world environments, especially when aiming to surpass human performance. The speaker details a modular approach combining state estimation, gate detection, and reinforcement learning, and discusses the critical role of sim-to-real transfer and adaptation to novelty, failure, and uncertainty. The presentation also delves into the unique strategies observed in human drone racing champions and how these interactions inform the definition of embodied intelligence.

Speakers

  • Antonio Loquercio — UPenn

Talks (1)

  • 00:00:00 — Antonio Loquercio: What does Embodied Intelligence mean? Lessons Learned from Drone Racing
    • A discussion on embodied intelligence, focusing on the challenges and insights gained from developing an autonomous drone racing system that can compete with human champions.

Key Takeaways

  • Embodied intelligence is defined as the ability to deal with novelty, failure, and uncertainty, particularly with limited data and computational resources.
  • Traditional imitation learning is often insufficient for tasks requiring superhuman performance; reinforcement learning or optimization-based methods are more suitable.
  • Sim-to-real transfer is a significant challenge, requiring careful modeling of real-world latencies, dynamics, and sensor errors, often through iterative data collection and simulator improvement.
  • Interaction with the environment and other agents (even adversarial ones) provides crucial learning opportunities, highlighting the importance of robust and adaptive systems.
  • Self-supervised learning, particularly predicting one sensor from another (e.g., vision predicting proprioception), can be a powerful tool for agents to learn about their environment and adapt to novel conditions.

Methods / Models / Datasets Mentioned

  • VIO
  • Kalman filter
  • MLP policy
  • Reinforcement Learning
  • Domain Randomization
  • MPC
  • RL
  • CLIP encoder
  • Rectified Flow Model

Topics

Embodied Intelligence · Drone Racing · Autonomous Systems · Reinforcement Learning · Sim-to-Real Transfer · Perception Latency · Optimization · Self-Supervised Learning · Legged Locomotion · Cross-Modal Supervision

Notes

Open for commentary — connections to other work, critiques, follow-up reading.