Home/Looking Forward: A Semantic Mapping System for Scouting with Micro-Aerial Vehicles

Looking Forward: A Semantic Mapping System for Scouting with Micro-Aerial Vehicles

Daniel Maturana, Sankalp Arora and Sebastian Scherer
Carnegie Mellon University, International Conference on Intelligent Robots and Systems (IROS), September, 2017

Download Publication (PDF)

Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author’s copyright. These works may not be reposted without the explicit permission of the copyright holder.


The last decade has seen a massive growth in applications for Micro-Aerial Vehicles (MAVs), due in large part to their versatility for data gathering with cameras, LiDAR and various other sensors. Their ability to quickly go from assessing large spaces from a high vantage point to flying in close to capture high-resolution data makes them invaluable for applications where we are interested in a specific target with ana priori unknown location, e.g. survivors in disaster response scenarios, vehicles in surveillance, animals in wildlife monitoring, etc., a task we will refer to scouting. Our ultimate goal is to enable MAVs to perform autonomous scouting. In this paper, we describe a semantic mapping system designed to support this goal. The system maintains a 2.5D map describing its belief about the location of semantic classes of interest, using forward-looking cameras and state estimation. The map is continuously updated on the fly, using only onboard processing. The system couples a deep learning 2Dsemantic segmentation algorithm with a novel mapping method to project and aggregate the 2D semantic measurements into a global 2.5D grid map. We train and evaluate our segmentation method on a novel dataset of cars labeled in oblique aerial imagery. We also study the performance of the mapping system in isolation. Finally, we show the integrated system performing a fully autonomous car scouting mission in the field.

BibTeX Reference
title = {Looking Forward: A Semantic Mapping System for Scouting with Micro-Aerial Vehicles},
author = {Daniel Maturana and Sankalp Arora and Sebastian Scherer},
booktitle = {International Conference on Intelligent Robots and Systems (IROS)},
keyword = {semantic segmentation, semantic mapping, micro-= aerial vehicle, unmanned aerial vehicle},
sponsor = {Office of Naval Research, Qualcomm},
school = {Robotics Institute , Carnegie Mellon University},
month = {September},
year = {2017},
address = {Pittsburgh, PA},