/Supervised Learning of Corrective Maneuvers for Vision-Based Autonomous Flight

Supervised Learning of Corrective Maneuvers for Vision-Based Autonomous Flight

Dhruv Mauria Saxena
Master's Thesis, Tech. Report, CMU-RI-TR-17-55, Robotics Institute, Carnegie Mellon University, August, 2017

Download Publication (PDF)

Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author’s copyright. These works may not be reposted without the explicit permission of the copyright holder.


The ability of autonomous mobile robots to react to and recover from potential failures of on-board systems is an important area of ongoing robotics research. With increasing emphasis on robust systems and safe navigation, mobile robots must be able to respond safely and intelligently to dangerous situations. Recent developments in computer vision have made autonomous vision based navigation possible. However, vision systems are known to be imperfect and prone to failure due to variable lighting, terrain changes, and other environmental variables. The notion of introspection for mobile robots has been developed recently which provides autonomous agents with a self-evaluating capability. This allows them to assess the quality of decisions made by them in the future based on the present input given or available to them. This thesis focuses on the situation when an agent is in a situation where the input available is unreliable (for whatever reason), and therefore any action taken using that input will also likely be unreliable. In this paradigm, we propose two different solutions.

First, we describe a system for learning simple failure recovery maneuvers based on experience. A failure instance is one where the input data is unreliable. This involves both recognizing when the vision system is prone to failure, and associating failures with appropriate responses that will most likely help the robot recover. We implement this system on an autonomous quadrotor and demonstrate that behaviors learned with our system are effective in recovering from situational perception failure, thereby improving reliability in cluttered and uncertain forest environments.

While the first solution only looks at recovering after a failure has been detected, we also consider the case where we pre-emptively avoid failures by proactively executing a `recovery’ maneuver if our system believes that for the current input, reliable or not, it will improve performance. This is essentially a continuous case extension of the previous solution which looked at discrete changes between a non-failure and failure mode. Again, we evaluate our performance on an autonomous quadrotor in flight through a outdoor forest environment.

BibTeX Reference
author = {Dhruv Mauria Saxena},
title = {Supervised Learning of Corrective Maneuvers for Vision-Based Autonomous Flight},
year = {2017},
month = {August},
school = {Carnegie Mellon University},
address = {Pittsburgh, PA},
number = {CMU-RI-TR-17-55},
keywords = {Failure Detection and Recovery, Visual-Based Navigation, Aerial Robotics, Cost-Sensitive Classification, Supervised Learning},