/Grounding Spatial Relations for Outdoor Robot Navigation

Grounding Spatial Relations for Outdoor Robot Navigation

Abdeslam Boularias, Felix Duvallet, Jean Hyaejin Oh and Anthony (Tony) Stentz
Conference Paper, 2015 IEEE International Conference on Robotics and Automation (ICRA), May, 2015

Download Publication (PDF)

Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author’s copyright. These works may not be reposted without the explicit permission of the copyright holder.


We propose a language-driven navigation approach for commanding mobile robots in outdoor environments. We consider unknown environments that contain previously unseen objects. The proposed approach aims at making interactions in human-robot teams natural. Robots receive from human teammates commands in natural language, such as “Navigate around the building to the car left of the fire hydrant and near the tree”. A robot needs first to classify its surrounding objects into categories, using images obtained from its sensors. The result of this classification is a map of the environment, where each object is given a list of semantic labels, such as “tree” and “car”, with varying degrees of confidence. Then, the robot needs to ground the nouns in the command. Grounding, the main focus of this paper, is mapping each noun in the command into a physical object in the environment. We use a probabilistic model for interpreting the spatial relations, such as “left of” and “near”. The model is learned from examples provided by humans. For each noun in the command, a distribution on the objects in the environment is computed by combining spatial constraints with a prior given as the semantic classifier’s confidence values. The robot needs also to ground the navigation mode specified in the command, such as “navigate quickly” and “navigate covertly”, as a cost map. The cost map is also learned from examples, using Inverse Optimal Control (IOC). The cost map and the grounded goal are used to generate a path for the robot. This approach is evaluated on a robot in a real-world environment. Our experiments clearly show that the proposed approach is efficient for commanding outdoor robots.

BibTeX Reference
author = {Abdeslam Boularias and Felix Duvallet and Jean Hyaejin Oh and Anthony (Tony) Stentz},
title = {Grounding Spatial Relations for Outdoor Robot Navigation},
booktitle = {2015 IEEE International Conference on Robotics and Automation (ICRA)},
year = {2015},
month = {May},