Explore and Exploit: Learning Policies for Efficient and Coordinated Active Search - Robotics Institute Carnegie Mellon University
Loading Events

MSR Thesis Presentation

May

26
Tue
Arsh Verma MSR Student Robotics Institute,
Carnegie Mellon University
Tuesday, May 26
1:30 pm to 2:30 pm
Newell-Simon Hall 4305
Explore and Exploit: Learning Policies for Efficient and Coordinated Active Search

Abstract:

Robotic search is becoming a central capability in domains where the world is large, uncertain, and costly to inspect directly: search and rescue, environmental monitoring, surveillance, and infrastructure inspection. In these settings, the hard problem is not perception alone but the online sensing decision: where to look next as evidence arrives, while every motion command spends travel distance and mission time. Active search requires both exploration and exploitation: reducing uncertainty across the searchable region, and using the emerging posterior to confirm likely objects of interest. This thesis studies that decision problem under a shared Bayesian active-search formulation, and develops learned policies for real-time field execution and coordination across heterogeneous teams.

The first contribution is a path-aware single-robot policy. We introduce a path-integral expert that scores complete shortest-path routes by their accumulated expected information gain, then amortize that expensive expert into a Graph Attention Network policy by behavior cloning. The resulting policy achieves up to a 254x speedup over the path-integral expert and an 11x speedup over a one-step greedy expert, generalizes across map geometries and object densities without retraining, and yields up to 2.86x faster mission completion against a strong NATS baseline in field tests on an autonomous ground vehicle in a 75,000 square meter forested environment.

The second contribution extends the same shared-belief interface from single-robot search to coordinated teams. We propose QUEST, a multi-agent Q-learning framework that trains over the same graph backbone but optimizes a per-decision Bellman target over downstream belief, robot positions, team coverage, and remaining budget. The formulation covers single robots, homogeneous UGV teams, and heterogeneous UAV-UGV teams. QUEST reaches 0.913 F-score with four UGVs versus 0.853 for the strongest learned baseline, maintains its lead under communication outage and two simultaneous mid-mission robot failures without retraining, and on out-of-distribution UAV-UGV compositions improves early F-score, 0.759 versus 0.728, while using 10% less path length and 12% less duplicate coverage. A map-structure analysis using algebraic connectivity and modularity ties these gains to the topology of the search graph, predicting when non-myopic value learning is and is not worth its training cost.

Together, these policies connect field deployment with coordinated team search that generalizes across composition, communication, and partial failure under one shared-belief active-search formulation.

Committee:
Dr. Jeff Schneider (chair)
Dr. Wennie Tabib
Dr. Tejus Gupta