We argue that the key challenge in deploying robots in unstructured environments is the difficulty of interpreting high-dimensional streams of multi-modal perception data, and translating them into robust, expressive representations for downstream path planning and control. This problem is exacerbated by the fact that unstructured environments are highly uncertain and lack a clean mapping from semantics to navigation decisions (i.e. traversability).

While there have been notable successes in deploying robots in unstructured environments, such systems are highly tuned to a given robot platform and environment, and are the result of months to years of dedicated effort from a highly skilled team of engineers. As such, deployment of these systems to new platforms or environments requires extensive tuning of perception-planning interfaces, labeling of semantic images, etc. Unfortunately, this makes engineering expertise the bottleneck in the widespread adoption of mobile robots.

I will present my work in designing a learning-based perception and traversability system for high-speed off-road driving from camera and LiDAR data. Importantly, this approach is annotation-free, enabling performance matching or exceeding current SoTA in off-road driving from a relatively small amount of tele-operated demonstrations.