From Sim2Real 1.0 to 4.0 for Humanoid Whole-Body Control and Loco-Manipulation

Event: CVPR 2025 · Duration: 34 min · ▶ Watch on YouTube

Abstract

This presentation explores the progression of Sim2Real methodologies for humanoid whole-body control and loco-manipulation, from traditional model-based control (Sim2Real 1.0) to reinforcement learning (Sim2Real 2.0) and advanced hybrid techniques (Sim2Real 3.0). The speaker highlights the limitations of teleoperation and video-based learning for tasks requiring whole-body agility, emphasizing the need for simulator-based learning. Key works like OmniH2O, ASAP, and FALCON are introduced, demonstrating real-time human-to-humanoid motion translation, agile control, and force-adaptive loco-manipulation. The talk concludes with a forward-looking perspective towards Sim2Real 4.0, advocating for better models, advanced RL algorithms, and improved online reasoning to unlock true humanoid dexterity and agility.

Speakers

  • Guanya Shi — Assistant Professor, Robotics Institute, CMU

Talks (1)

  • 00:04 — Guanya Shi: From Sim2Real 1.0 to 4.0 for Humanoid Whole-Body Control and Loco-Manipulation
    • This talk presents an evolution of Sim2Real techniques from 1.0 to 3.0, covering model-based control, reinforcement learning, and hybrid approaches, and outlines a vision for Sim2Real 4.0 in achieving agile humanoid control and loco-manipulation.

Key Takeaways

  • Tasks requiring whole-body agility are difficult for traditional teleoperation or video-based imitation learning due to challenges in obtaining labeled actions and the limitations of simple low-level controllers.
  • Sim2Real 2.0 (Reinforcement Learning) offers a powerful approach by training policies offline in massively parallel simulated environments, eliminating the need for state estimation and enabling robust real-world deployment.
  • Sim2Real 3.0 introduces hybrid approaches like Real2Sim (learning delta action models to bridge the sim2real gap) and Structured RL (leveraging humanoid structure for better policy architectures) to address the limitations of pure RL.
  • Directly learning residual dynamics for humanoids is challenging due to generalization requirements, regularization needs, and the potential for RL to exploit learned dynamics.
  • The future of Sim2Real (4.0) lies in combining better models (e.g., generative sims), advanced RL algorithms, and improved online reasoning, potentially through powerful offline+online learning paradigms.

Methods / Models / Datasets Mentioned

  • OmniH2O
  • ASAP
  • FALCON
  • Physical Intelligence π0.5
  • Tesla Optimus
  • Humanoid Policy ~ Human Policy
  • Inverted Pendulum Model
  • Single Rigid Body Model
  • Online Model Predictive Control (MPC)
  • DIAL-MPC
  • NVIDIA Isaac
  • Genesis
  • MuJoCo
  • Proximal Policy Optimization (PPO)
  • AMASS
  • Inverse Kinematics
  • GPT-4o
  • HOOVER
  • VideoMimic
  • WoCoCo
  • Neural-Lander
  • Neural-Fly

Topics

Sim2Real · Humanoid Control · Loco-Manipulation · Reinforcement Learning · Model Predictive Control · Whole-Body Agility · Teleoperation · Residual Dynamics Learning · Structured RL · Delta Action Model

Notes

Open for commentary — connections to other work, critiques, follow-up reading.