Inferring Maps and Behaviors from Natural Language Instructions - Robotics Institute Carnegie Mellon University

Inferring Maps and Behaviors from Natural Language Instructions

Felix Duvallet, Matthew Walter, Thomas Howard, Sachithra Hemachandra, Jean Hyaejin Oh, Seth Teller, Nicholas Roy, and Anthony (Tony) Stentz
Conference Paper, Proceedings of 14th International Symposium on Experimental Robotics (ISER '14), pp. 373 - 388, June, 2014

Abstract

Natural language provides a exible, intuitive way for people to command robots, which is becoming increasingly important as robots transition to working alongside people in our homes and workplaces. To follow instructions in unknown environments, robots will be expected to reason about parts of the environments that were described in the instruction, but that the robot has no direct knowledge about. However, most existing approaches to natural language understanding require that the robot's environment be known a priori. This paper proposes a probabilistic framework that enables robots to follow commands given in natural language, without any prior knowledge of the environment. The novelty lies in exploiting environment information implicit in the instruction, thereby treating language as a type of sensor that is used to formulate a prior distribution over the unknown parts of the environment. The algorithm then uses this learned distribution to infer a sequence of actions that are most consistent with the command, updating our belief as we gather more metric information. We evaluate our approach through simulation as well as experiments on two mobile robots; our results demonstrate the algorithm's ability to follow navigation commands with performance comparable to that of a fully-known environment.

BibTeX

@conference{Duvallet-2014-7894,
author = {Felix Duvallet and Matthew Walter and Thomas Howard and Sachithra Hemachandra and Jean Hyaejin Oh and Seth Teller and Nicholas Roy and Anthony (Tony) Stentz},
title = {Inferring Maps and Behaviors from Natural Language Instructions},
booktitle = {Proceedings of 14th International Symposium on Experimental Robotics (ISER '14)},
year = {2014},
month = {June},
pages = {373 - 388},
}