Learning Models and Cost Functions from Unlabeled Data for Off-Road Navigation

Abstract

Off-road driving is an important instance of navigation in unstructured environments, which is a key robotics problem with many applications, such as exploration, agriculture, disaster response and defense. The primary challenge in off-road driving is to be able to take in high dimensional, multi-modal sensing data and use it to make intelligent decisions on where and how to traverse over unstructured terrain. While off-road driving has been the focus of much research, it remains challenging to design systems that are capable of off-road driving quickly, robustly, and in a variety of environments.

It is common practice for modern off-road driving systems to perform some form of semantic segmentation to bin terrain into one of several discrete classes (such as trails, vegetation, obstacle) and perform path planning and trajectory optimization to navigate through low-cost terrain classes. While such systems have generated strong empirical results, they are often engineering intensive, requiring thousands of densely labeled images. Such a process can take tens to hundreds of hours of an engineer's time. Furthermore, the mapping of terrain classes to cost ignores geometric information (e.g. some bushes may be more dangerous than others), and the relative cost of different terrain classes is not defined. As a result, these systems often need to be tuned extensively in the field, which generally requires the effort of many skilled engineers.

In order to avoid this engineering-intensive labeling and tuning process, recent work has proposed learning end-to-end navigation policies from autonomously collected data. While such an approach has the promise of reducing the engineering burden to deploy off-road driving systems, these approaches often lack interpretability and performance guarantees. Additionally, relying on random exploration to collect training data is unsafe for full-scale robots, which can seriously damage the environment and themselves. As such, these methods are generally infeasible to deploy on full-scale robots.

In this thesis, we propose a learning-based system for off-road driving for a full-scale autonomous all-terrain vehicle. Key to the work in this these are two assertions: 1) We can design a high-performance system without the need of any human-annotated data, and 2) learning should be used in concert with existing trajectory optimization methods for off-road driving. To address the first point, we collect a large dataset containing several hours of human-driven trajectories through challenging off-road terrain at high speeds. This dataset contains many traversability-stressing scenarios, such as steep slopes, dense vegetation and high speeds. To address the second point, we leverage this dataset to learn two key functions in trajectory optimization, the dynamics model and the cost function.

Overall, we find that our learning-based methods outperform traditional common-practice navigation baselines in isolation. More importantly, we also demonstrate that these models lead to improved performance when incorporated in path planning and control algorithms through large-scale, multi-kilometer navigation trials.