AI4Space 2024 Workshop

Event: AI4Space Workshop, CVPR 2024, June 17-21, Seattle, WA · Duration: 127 min · ▶ Watch on YouTube

Abstract

The segment covers the opening remarks of the AI4Space 2024 workshop, including an overview of the SPARK Challenge, paper statistics, and program details. It then features several spotlight presentations on topics such as robust crater-based camera pose estimation, AI-based satellite pose estimation using synthetic datasets, Martian dust displacement detection, mitigating space environment challenges for onboard AI, dual-mode vision-based navigation for lunar landing, and tackling the satellite downlink bottleneck with federated onboard learning of image compression. This segment features a presentation on “AI in space, what for?” by Gianluca Furano from ESA/ESTEC, highlighting the challenges and opportunities of integrating AI into space missions. It covers the need for enhanced onboard processing, data standardization, and agile inference networks to enable low-latency services and unlock the significant economic value of Earth Observation data. The segment also includes the awards ceremony for the SPARK 2024 challenge and the AI4Space 2024 workshop, recognizing top performers in spacecraft segmentation and trajectory estimation, as well as best papers. Finally, it provides information on upcoming related conferences and a concluding thank you from the organizers.

Speakers

  • Tat-Jun Chin — The University of Adelaide
  • Gabriele Meoni — TU Delft
  • Djamila Aouada — University of Luxembourg
  • Tae Ha “Jeff” Park — Stanford’s Space Rendezvous Lab
  • Rajat Talak — MIT
  • Viorela Ila — The University of Sydney
  • Nicolas Longepé — European Space Agency
  • Arunkumar Rathinam — University of Luxembourg
  • Soon-Jo Chung — California Institute of Technology
  • Sofia McLeod — The University of Adelaide
  • Fabien Gallet — IRT Saint Exupéry, Thales Alenia Space
  • Ana Lomashvili — DLR
  • Jonathan Morgan — Melbourne Space Laboratory, The University of Melbourne
  • Roberto Del Prete — University of Naples Federico II, Telespazio SRL, e-lab, ESA ESRIN
  • Gianluca Furano — ESA/ESTEC
  • Djemila Ouarada — ESA

Talks (10)

  • 00:05:05 — Soon-Jo Chung: Contraction Is All You Need in Learning for Space
    • Discusses deep learning limitations in robotics, introduces Neural-Fly for flight control in strong winds, standardized autonomy hardware, interstellar object rendezvous, multi-spacecraft reconfiguration, chance-constrained optimal control, contraction theory for stability, and distributed estimation.
  • 00:41:00 — Djamila Aouada: SPARK 2024 CHALLENGE
    • Overview of the SPARK 2024 Challenge, including timeline, streams (spacecraft segmentation, trajectory estimation), data sources (synthetic and lab), organizers, and prizes.
  • 00:41:00 — Sofia McLeod: ROBUST PERSPECTIVE-N-CRATER FOR CRATER-BASED CAMERA POSE ESTIMATION
    • Presents a robust PnC approach for 6DoF camera pose estimation from crater correspondences, minimizing reprojection error using an elliptical format and M-estimators, and introduces the CRESENT dataset.
  • 00:41:15 — Fabien Gallet: Exploring AI-Based Satellite Pose Estimation: from Novel Synthetic Dataset to In-Depth Performance Evaluation
    • Introduces a new synthetic dataset (RAPTOR) of 120,000 images for space rendezvous navigation, detailing its features, checks, and two domains (nominal and disturbed) for evaluating pose estimation solutions.
  • 00:45:29 — Ana Lomashvili: OPTIMIZED MARTIAN DUST DISPLACEMENT DETECTION USING EXPLAINABLE MACHINE LEARNING
    • Presents a two-stage architecture for detecting Martian dust displacement using ChemCam images from the Curiosity rover, combining VGG16 for feature extraction with a Random Forest classifier, and explores explainable AI techniques for feature importance.
  • 00:49:44 — Jonathan Morgan: Mitigating Challenges of the Space Environment for Onboard Artificial Intelligence: Design Overview of the Imaging Payload on SpIRIT
    • Discusses challenges for deploying AI in space for small-scale nanosatellites, focusing on radiation, thermal management, and limited transmission bandwidth, and presents the Loris imaging payload on SpIRIT.
  • 00:54:34 — Roberto Del Prete: A Dual-Mode Approach for Vision-Based Navigation in a Lunar Landing Scenario
    • Proposes a dual-mode vision-based navigation approach for lunar landing, combining absolute (crater detection/matching) and relative (CNN/GNN feature detection/matching) modes, and presents simulation results using an Unreal Engine simulator.
  • 00:59:28 — Gabriele Meoni: Tackling the Satellite Downlink Bottleneck with Federated Onboard Learning of Image Compression
    • Addresses the satellite downlink bottleneck problem for Earth Observation missions, proposing the LICOS approach (Learning Image Compression Onboard Satellites) using federated learning and autoencoders to train image compression models on a constellation of satellites.
  • 01:03:24 — Gianluca Furano: AI in space, what for?
    • Discusses the challenges and opportunities of deploying AI in space, emphasizing the need for robust data processing, standardization, and increased onboard capabilities to enable low-latency services and unlock significant economic value from Earth Observation data.
  • 01:07:02 — Djemila Ouarada: Awards Ceremony
    • Hosts the awards ceremony for the SPARK 2024 challenge and the AI4Space 2024 workshop, announcing winners for Spacecraft Segmentation, Spacecraft Trajectory Estimation, Best Presentation, Best Paper (Honorable Mention), and Best Paper.

Key Takeaways

  • The AI4Space 2024 workshop covers a broad range of topics in artificial intelligence for space applications, from fundamental research in robotics and control to practical challenges in satellite operations.
  • Federated learning and onboard processing are crucial for addressing the satellite downlink bottleneck, enabling efficient data compression and selective data transmission.
  • Robust pose estimation and navigation techniques, utilizing both synthetic and real-world data, are essential for autonomous spacecraft operations, including rendezvous, docking, and lunar landings.
  • Addressing the unique challenges of the space environment, such as radiation, thermal extremes, and limited communication bandwidth, is critical for the successful deployment and operation of AI systems in orbit.
  • AI is crucial for enabling low-latency services and maximizing the economic value derived from Earth Observation data.
  • Significant hurdles for AI adoption in space include data standardization, processing capabilities, agile inference networks, and mass memory expansion.
  • The future of space data utilization involves a shift from delivering raw pixels to providing actionable insights and solutions directly to end-users through AI-driven processing.
  • Organizational structures and risk-averse cultures within space agencies pose a challenge to integrating new AI technologies, despite their potential.

Methods / Models / Datasets Mentioned

  • A. P. Dani, S.-J. Chung, and S. Hutchinson, IEEE Transactions on Automatic Control, 2015
  • ADMM (Alternating Direction Method of Multipliers)
  • AlphaGo
  • AlphaZero
  • Autoencoder
  • Benjamin Rivière, John Lathrop, and Soon-Jo Chung, Bringing Games to Life: Decision-Making for Dynamical Systems with Spectral Expansion Tree Search
  • CNN
  • CNN (Convolutional Neural Network)
  • CRESENT Dataset
  • ChemCam
  • Christian et al. 2021
  • CompressAI framework
  • Contraction Theory
  • Curiosity Rover
  • D. Morgan et al., The International Journal of Robotics Research, 2016
  • D. Morgan, Chung, Hadaegh, JGCD 2014
  • Del Prete et al. 2022
  • Downes et al. 2020
  • E.S. Lupu, F. Xie et al., Meta-Learning Adaptation for Ground Interaction Control with Visual Foundation Models, under review
  • EKF (Extended Kalman Filter)
  • Epipolar Geometry
  • Essential Matrix Estimation
  • FLIR Lepton 3.5
  • Faster-RCNN
  • Federated Learning
  • GAN
  • GNN (Graph Neural Network)
  • Guided Backpropagation
  • Intel Neural Compute Stick 2 (NCS2)
  • J. Yang, A. Dai, S.-J. Chung, and S. Hutchinson, Journal of Field Robotics, 2017
  • JPEG-XL
  • Johnson et al. 2008
  • K. Matsuka and S.-J. Chung, IEEE Transactions on Robotics (2023)
  • K. Matsuka and S.-J. Chung, IEEE Transactions on Robotics, October 2023
  • K. Matsuka, et al., Advances in Space Research, vol. 67, no. 11, June 2021
  • K. Meier, S.-J. Chung, and S. Hutchinson, Journal of Field Robotics, 2018
  • Kaess, Michael, et al. iSAM2: Incremental smoothing and mapping using the Bayes tree. IJRR. 31.2 (2012)
  • LIBS (Laser-Induced Breakdown Spectroscopy)
  • LICOS (Learning Image Compression Onboard Satellites)
  • Landmark Regression Network
  • Loris (Imaging and AI Payload)
  • MPL
  • Michael O'Connell, Joshua Cho, Matthew Anderson, and Soon-Jo Chung, Learning-Based Minimally-Sensed Fault-Tolerant Adaptive Flight Control
  • Monte Carlo Tree Search (MCTS)
  • NVIDIA Jetson Nano
  • Nakka and Chung, IEEE Transactions on Robotics, 2023
  • Neural-Fly
  • O'Connell & G. Shi et al., Science Robotics, 2022
  • PANGU (Planet and Asteroid Natural scene Generation Utility)
  • PNC (Perspective-n-Crater)
  • Paseos (Power, Attitude, and Sensor Effects on Satellite)
  • PnP (Perspective-n-Point)
  • Power-Temperature-Limited-Line-of-Sight (PASEOS)
  • R. Foust, Chung, Hadaegh, JGCD 2019
  • RAPTOR Dataset
  • RMI (Remote Micro-Imager)
  • RNN
  • Random Forest
  • ResNet-50
  • S. Bandyopadhyay and S.-J. Chung, Automatica, 2018
  • SETS (Spectral Expansion Tree Search)
  • SPARK Challenge
  • SVM
  • Sentinel-2
  • Shapley values
  • SpIRIT (Space Industry Responsive Intelligent Thermal)
  • SuperGlue
  • SuperPoint
  • THRaWS dataset
  • Tsukamoto and Chung, IEEE CSS-L
  • Tsukamoto, Chung, Slotine, Annual Reviews in Control, 2021
  • Tsukamoto, Chung, Slotine, Automatica 1998
  • UKF
  • Ubotica CogniSat-XE1 (XE1)
  • Unreal Engine Simulator
  • V. Capuano, A. Harvard, and S.-J. Chung, Progress in Aerospace Sciences, vol. 128, January 2022
  • VGG16
  • iDFGO (Incremental Distributed Factor Graph Optimization)
  • iSAM2 (Incremental Smoothing and Mapping 2)
  • tCNN

Topics

AI challenges · AI in space · Deep Learning · Explainable AI · Fault Tolerance · Federated Learning · Image Compression · Lunar Navigation · Robotics · Satellite Pose Estimation · Space AI · Space Environment Challenges · Spacecraft Autonomy · awards ceremony · conference announcements · data standardization · disintermediation · economic value of EO data · inference networks · low-latency services · mass memory expansion · onboard processing · spacecraft segmentation · trajectory estimation

Notes

Open for commentary — connections to other work, critiques, follow-up reading.