CVPR Tutorial June 2024: Deep Learning for Camera Physiological Measurement

Event: CVPR 2024 Tutorial · Duration: 59 min · ▶ Watch on YouTube

Abstract

This tutorial introduces deep learning techniques for camera-based physiological measurement. It covers the principles behind extracting vital signs like heart rate, breathing, and blood pressure from video, discussing the underlying optical models and photoplethysmography (PPG) waveforms. The presentation highlights the challenges of limited real-world datasets and explores solutions through synthetic data generation and data augmentation techniques. It also delves into various deep learning models, including convolutional neural networks and self-supervised learning approaches, for robust and accurate vital sign estimation.

Speakers

  • Daniel McDuff — Affiliate Prof. University of Washington, Google

Talks (1)

  • 00:00:00 — Daniel McDuff: Deep Learning for Camera Physiological Measurement
    • An overview of deep learning techniques for camera-based physiological measurement, covering principles, data, models, optimization, and evaluation.

Key Takeaways

  • Camera-based physiological measurement leverages subtle changes in skin color due to blood flow, offering a non-contact method for vital sign monitoring.
  • Deep learning models, particularly CNNs and RNNs, are crucial for extracting accurate physiological signals from video, often employing motion and appearance branches.
  • Data scarcity is a major challenge, addressed by synthetic data generation (e.g., SCAMPS dataset) and various data augmentation techniques (motion, speed, spatial, color space).
  • Self-supervised learning methods like SimPer show promise in improving representation learning for periodic targets, enhancing model generalization and robustness.
  • Labels matter significantly; models trained on finger-based PPG may not generalize well to face-based PPG due to morphological differences.

Methods / Models / Datasets Mentioned

  • Shafer's Dichromatic Reflection Model (DRM)
  • Independent Component Analysis (ICA)
  • Chrominance (CHROM)
  • Local Grouping Index (LGI)
  • Plane Orthogonal to Skin (POS)
  • Principal Component Analysis (PCA)
  • Green Channel (GREEN)
  • Pulse-Blood Volume (PBV)
  • DeepPhys
  • PhysNet
  • TS-CAN
  • EfficientPhys
  • Hybrid-CAN
  • SimPer
  • MoCo v2
  • SimCLR
  • CVRL
  • BYOL
  • SCAMPS Dataset
  • UBFC-rPPG Dataset
  • PURE Dataset
  • AFRL Dataset
  • MMSE-HR Dataset
  • MR-NIRP Dataset
  • MAHNOB-HCI Dataset
  • BP4D Dataset
  • VIPL-HR Dataset
  • COHFACE Dataset
  • UBFC-PHYS Dataset
  • RICE CameraHRV Dataset
  • OBF Dataset
  • PFF Dataset
  • VicarrPPG Dataset
  • VicarrPPG-2/CleanerPPG Dataset
  • CMU Dataset
  • MMPD Dataset
  • rPPG-Toolbox
  • iPhys Toolbox

Topics

Camera Physiological Measurement · Deep Learning · Photoplethysmography (PPG) · Vital Signs · Synthetic Data Generation · Data Augmentation · Self-Supervised Learning · Model Architectures

Notes

Open for commentary — connections to other work, critiques, follow-up reading.