Abstract:
Off-road autonomous driving presents significant challenges such as navigating unmapped variable environments, traversing difficult terrain geometries such as steep slopes and ditches, and managing complex terrain dynamics. Addressing these challenges requires effective low-level adaptable control and long-horizon planning. Most existing methods utilize Model Predictive Control (MPC) methods such as Model Predictive Path Integral (MPPI), which have long-horizon planning capabilities, but require expensive dense sampling and precise dynamics modeling that are impractical to deploy for real-time control. On the other hand, Reinforcement Learning (RL) learns complex dynamics and reactive low-level control policies directly from interaction, but typically fails to plan and navigate dense environments due to poor exploration.

To utilize the different strengths of both MPC methods and RL, this thesis proposes a hierarchical pipeline with a low-frequency high-level MPPI planner for long-horizon planning and a high-frequency low-level adaptable RL controller. In order to solve RL’s poor exploration problem, this thesis introduces Teacher Action Distillation Policy Optimization (TADPO), a novel teacher-student policy gradient formulation that extends Proximal Policy Optimization (PPO) to guide policy learning, leveraging off-policy teacher trajectories for teacher guidance and on-policy trajectories for student exploration.

With TADPO, we develop a vision-based, end-to-end RL policy and off-road autonomy system for high-speed driving, capable of navigating extreme slopes and obstacle-dense terrain. Firstly, we demonstrate our performance in simulation and, more importantly, zero-shot sim-to-real transfer on a full-scale off-road vehicle. Then, we show further performance improvements of the deployed policy in simulation. Finally, we illustrate TADPO’s generalizability with a policy that navigates through trees in simulation. To our knowledge, this work represents the first deployment of RL-based policies on a full-scale off-road robotics platform.

Committee:
Jeff Schneider (advisor)
John Dolan
Samuel Triest