CVPR 2024 - Invited Speakers - Chris Padwick

Event: CVPR 2024 · Duration: 441 min · ▶ Watch on YouTube

Abstract

This video segment features Chris Padwick introducing the challenges facing agriculture, including a growing global population, extreme climate variability, and an aging workforce, emphasizing the critical need for automation and autonomy. He then introduces Ranveer Chandra from Microsoft, who presents his vision for data-driven agri-food systems. Chandra details Microsoft’s initiatives in leveraging AI and machine learning to address these challenges, focusing on improving connectivity, enhancing spatial coverage with aerial imagery, and enabling micro-climate forecasting and pest detection. He also highlights the potential of generative AI through the “Agri CoPilot” to provide actionable insights for farmers and the broader agricultural ecosystem, demonstrating its ability to pass agronomy exams and learn new knowledge through fine-tuning. This segment features three talks from the CVPR 2024 Agriculture-Vision workshop. Paula Ramos from Intel discusses the application of AI at the edge in agriculture, highlighting the OpenVINO toolkit and AnomaliB library for anomaly detection. Muhammad Hamza Asad from the University of Regina presents an ensemble learning method for crop and weed detection in uncontrolled field conditions. Sambal Shikhar from Plaksha University introduces a label-free anomaly detection technique using masked image modeling for aerial agricultural images. This segment explores the evolution of training computer vision models, moving from traditional transfer learning with ImageNet pre-trained backbones to more advanced self-supervised learning techniques. It emphasizes the increasing availability of data and compute, which enables training models without extensive manual labeling. The speaker introduces active learning strategies, detailing how to select valuable data for annotation using predictions, embeddings, and metadata, and demonstrates significant accuracy gains in practical applications like plant detection and autonomous driving. This segment covers various aspects of applying computer vision and AI in agriculture. It highlights challenges such as data labeling bottlenecks, the need for efficient AI deployment on low-power edge devices, and the complexities of accurately monitoring soil carbon across diverse landscapes. Speakers present solutions involving active learning, quantization, model compression, and hybrid spatial-temporal modeling approaches. The segment also showcases winning solutions from an agricultural pattern recognition challenge, emphasizing techniques like ensemble models, pseudo-labeling, and data augmentation.

Speakers

  • Chris Padwick — Intelair, John Deere, Blue River Technology
  • Ranveer Chandra — Microsoft
  • Paula Ramos — Intel
  • Muhammad Hamza Asad — University of Regina, Regina Canada
  • Sambal Shikhar — Plaksha University
  • Igor Susmelj — Co-Founder Lightly.ai
  • Yuri Brigance — AIGEN
  • Matthew Harrison — University of Tasmania, Australia
  • Quan Quan — Alice Inc, Japan
  • Wang Lu — Hunan University, China
  • Zhao Qing Lu — Key Laboratory at Intelligent Perception & Image Understanding

Talks (12)

  • 00:00:00 — Chris Padwick: The Challenges of Agriculture
    • Discusses the challenges of agriculture including growing population, extreme variability due to climate change, and labor shortage/aging workforce, highlighting the need for automation and autonomy.
  • 00:05:08 — Ranveer Chandra: Data-driven Food Systems to Sustainably Nourish the World
    • Presents Microsoft’s vision for data-driven agri-food systems, addressing challenges in connectivity, sparse sensor deployments, and cloud integration, and highlighting the role of generative AI.
  • 01:28:13 — Paula Ramos: AGRICULTURE-VISION: CHALLENGES & OPPORTUNITIES FOR COMPUTER VISION IN AGRICULTURE The key in agriculture is the edge
    • Paula Ramos discusses the role of Intel’s OpenVINO toolkit and AnomaliB library in enabling AI applications at the edge for agriculture, emphasizing the need for optimized models and efficient data processing to address challenges like data imbalance and unknown abnormalities in real-world agricultural datasets.
  • 01:38:00 — Muhammad Hamza Asad: Improved Crop and Weed Detection with Diverse Data Ensemble Learning
    • Muhammad Hamza Asad presents a novel ensemble learning methodology for improved crop and weed detection under diverse and uncontrolled field conditions, leveraging multiple base models and a meta-teacher architecture to enhance performance and address the challenges of varying lighting, crop residue, and soil conditions.
  • 01:46:31 — Sambal Shikhar: Label-free Anomaly Detection in Aerial Agricultural Images with Masked Image Modeling
    • Sambal Shikhar presents a label-free anomaly detection approach using masked autoencoders for aerial agricultural images, addressing the challenges of diverse anomaly shapes and the difficulty of obtaining labeled data in real-world agricultural settings.
  • 02:56:42 — Igor Susmelj: Active Learning for Computer Vision
    • Igor Susmelj discusses the evolution of training computer vision models, highlighting the shift towards self-supervised learning and active learning to improve model performance and data efficiency.
  • 04:24:40 — Igor Susmelj: Lightly’s selection is 3.4x better than random
    • Igor Susmelj discusses Lightly’s active learning approach, demonstrating 3.4x better performance than random selection in data labeling for computer vision tasks in agriculture, leading to significant efficiency gains and addressing data bottlenecks.
  • 04:37:19 — Yuri Brigance: AIGEN ML Efficiency at the Edge CVPR 2024
    • Yuri Brigance presents AIGEN’s solar-powered autonomous robots for weeding in agriculture, highlighting the challenges of deploying ML models on ultra-low power edge devices (1.5W) without powerful GPUs, and their strategies for data collection, active learning, quantization, and model compression.
  • 05:13:28 — Matthew Harrison: Operationalising net-zero: what role can soil carbon play?
    • Matthew Harrison discusses the potential of soil carbon sequestration for net-zero goals, outlining challenges in accurate and cost-effective quantification using traditional methods, and presenting a hybrid approach combining spatial modeling (satellite imagery, machine learning) and temporal modeling (process-based models) to improve soil carbon prediction across large agricultural landscapes.
  • 05:35:20 — Quan Quan: 1st Place Winner’s Presentation
    • Quan Quan presents his winning solution for the agricultural pattern recognition challenge, utilizing an ensemble of models, pseudo-labeling, self-supervised learning, and test-time augmentation to achieve high accuracy in identifying agricultural features from aerial imagery.
  • 05:43:20 — Wang Lu: 2nd Place Winner’s Presentation
    • Wang Lu details his second-place solution, which employs multi-scale feature fusion, attention mechanisms, extensive data augmentation, and test-time augmentation to effectively address the agricultural pattern recognition task.
  • 05:51:20 — Zhao Qing Lu: 3rd Place Winner’s Presentation
    • The Key Laboratory team’s third-place solution leverages an ensemble of models, data augmentation, and test-time augmentation to achieve robust performance in the agricultural pattern recognition challenge.

Key Takeaways

  • The agricultural sector faces significant challenges including feeding a growing population, adapting to extreme climate variability, and overcoming labor shortages, necessitating advanced technological solutions like automation and AI.
  • Microsoft is developing “Data-driven Agri-Food systems” that leverage AI and machine learning across the entire food supply chain to improve efficiency, reduce costs, and ensure sustainability.
  • Key technological solutions include using TV White Space and satellites for farm connectivity, combining aerial imagery and AI for enhanced spatial coverage in sparse sensor deployments, and employing Azure Edge for local data processing and agricultural services.
  • Generative AI, exemplified by the “Agri CoPilot”, shows promise in providing actionable, context-aware insights to farmers, agronomists, and other stakeholders, even outperforming humans in some agricultural exams.
  • AI applications at the edge are crucial for agriculture, enabling real-time data processing, wider reach, and cost-efficiency, especially in areas with limited connectivity.
  • Ensemble learning methodologies can significantly improve crop and weed detection accuracy by combining knowledge from multiple base models, making them robust against diverse and uncontrolled field conditions.
  • Label-free anomaly detection using masked autoencoders offers a promising solution for agricultural imaging, reducing the reliance on extensive manual labeling while effectively identifying various types of anomalies.
  • Optimizing and quantizing AI models using tools like OpenVINO’s Neural Network Compression Framework (NNCF) is essential for deploying sophisticated AI applications on resource-constrained edge devices in agriculture.
  • Self-supervised learning on domain-specific data, combined with active learning, offers significant improvements in model performance and data efficiency compared to traditional transfer learning.
  • Active learning pipelines involve data ingestion, selection, labeling, training, and deployment, with continuous refinement through training loops.
  • Effective active learning requires flexible interfaces that integrate predictions, embeddings, and metadata to identify valuable and diverse data subsets for annotation.
  • In practical applications like plant detection and autonomous driving, active learning strategies can yield substantial accuracy gains (e.g., up to 10x) with fewer labeled images.
  • Active learning and efficient data curation are crucial for overcoming data labeling bottlenecks in agricultural AI, especially with large and diverse datasets.
  • Deploying AI on edge devices in agriculture requires significant optimization for low power consumption and limited computational resources, necessitating specialized hardware and quantization techniques.
  • Accurate soil carbon quantification is vital for net-zero goals but is challenging due to spatial-temporal variability and high measurement costs, requiring integrated modeling approaches.
  • Generative AI offers promising avenues for creating synthetic training data and assisting with labeling, particularly for rare events or difficult-to-collect scenarios in agriculture.

Methods / Models / Datasets Mentioned

  • ABE
  • APSIM
  • Active Learning
  • Agri CoPilot
  • AnomaliB
  • Artificial Neural Networks
  • Attention mechanism
  • Azure Edge
  • CP-decomposition
  • Computer Vision
  • Corset algorithm
  • DINO
  • DINOv2
  • DNDC
  • Data augmentation
  • Data curation
  • Data labeling
  • DayCent
  • DeepLab V3+
  • Diversity algorithm
  • Drone imagery
  • Ensemble models
  • Factorization (Candecomp/Parafac, Tucker decomposition)
  • FarmVibes.AI
  • Fine-tuning
  • Foundation Models
  • GPT-4
  • Generative AI
  • Generative Adversarial Networks (GAN)
  • HRNet
  • HSwish
  • ImageNet
  • Lightly framework
  • Llama 13B
  • MAE
  • MCE
  • MSNE
  • MUNE
  • Machine Learning
  • Mask Autoencoders
  • Model Compression
  • Multi-modal GPT
  • Multi-scale feature fusion
  • NNCF
  • ONNX
  • OpenVINO
  • PSPNet
  • Pseudo-labeling
  • PyTorch
  • Quantization (float32 vs int8)
  • Random Forest
  • ReLU6
  • Retrieval-Augmented Generation (RAG)
  • RothC
  • Satellite Communication
  • Satellite imagery (Landsat 8, PlanetScope, Sentinel-2)
  • SegFormer
  • SegNet
  • Self-supervised learning
  • Semantic Segmentation
  • SimCLR
  • Spatial modeling
  • TV White Space
  • Temporal modeling
  • TensorFlow
  • TensorFlow Lite
  • Test-time augmentation
  • Two-component decomposition
  • U-Net
  • Wavelet Transforms
  • YOLO
  • YOLOv8

Topics

AI in agriculture · Active Learning · Agricultural Robotics · Automation and Autonomy in Farming · Autonomous Driving · Challenges of Agriculture · Climate Change Impact · Computer Vision · Computer Vision in Agriculture · Data Curation · Data Efficiency · Data-driven Agri-Food Systems · Edge AI Deployment · Generative AI for Data Augmentation · Generative AI in Agriculture · Labor Shortage in Agriculture · Machine Learning Models · Microsoft FarmBeats · Model Compression · Model Quantization · Model Training · OpenVINO · Plant Detection · Precision Agriculture · Self-Supervised Learning · Soil Carbon Monitoring · anomaly detection · crop monitoring · edge computing · ensemble learning · masked autoencoders · weed detection

Notes

Open for commentary — connections to other work, critiques, follow-up reading.