Evaluation of Autonomous Ground Vehicle Skills

Abstract

Autonomous ground vehicles must achieve bold performance and solid reliability to mature from laboratory curiosities to fielded systems. Currently, there are no standard methods to measure, validate and compare the performance of autonomous unmanned ground vehicles, hence the impetus for this research. This paper documents the test methods implemented by Carnegie Mellon University?s Red Team while preparing robots for the DARPA Grand Challenge. The Red Team?s test methods were developed to enable quantitative evaluation of the effects of unit changes to the robots? hardware and software on the robots? over all ability to blindly track, track with perception assistance and modify based on perception a preplanned path. This paper describes tests for evaluating and comparing navigational skills of autonomous ground vehicles. Test data collected from the Red Team?s H1ghlander and Sandstorm autonomous unmanned ground vehicles is presented. Suggestions for future test methodology research and test standardization are also discussed.