Carnegie Mellon University
The
Spatial references and perspective in natural language instructions for collaborative manipulation

Shen Li, Rosario Scalise, Henny Admoni , Siddhartha Srinivasa, and Stephanie Rosenthal
IEEE International Symposium on Robot and Human Interactive Communication, August, 2016.


Download
  • Adobe portable document format (pdf) (2MB)
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Abstract
As humans and robots collaborate together on spatial tasks, they must communicate clearly about the objects they are referencing. Communication is clearer when language is unambiguous which implies the use of spatial references and explicit perspectives. In this work, we contribute two studies to understand how people instruct a partner to identify and pick up objects on a table. We investigate spatial features and perspectives in human spatial references and compare word usage when instructing robots vs. instructing other humans. We then focus our analysis on the clarity of instructions with respect to perspective taking and spatial references. We find that only about 42% of instructions contain perspective-independent spatial references. There is a strong correlation between participants’ accuracy in executing instructions and the perspectives that the instructions are given in, as well between accuracy and the number of spatial relations that were required for the instruction. We conclude that sentence complexity (in terms of spatial relations and perspective taking) impacts understanding, and we provide suggestions for automatic generation of spatial references.

Notes
Associated Center(s) / Consortia: Quality of Life Technology Center, National Robotics Engineering Center, and Center for the Foundations of Robotics
Associated Lab(s) / Group(s): Personal Robotics
Number of pages: 8

Text Reference
Shen Li, Rosario Scalise, Henny Admoni , Siddhartha Srinivasa, and Stephanie Rosenthal, "Spatial references and perspective in natural language instructions for collaborative manipulation," IEEE International Symposium on Robot and Human Interactive Communication, August, 2016.

BibTeX Reference
@inproceedings{Li_2016_8208,
   author = "Shen Li and Rosario Scalise and Henny {Admoni } and Siddhartha Srinivasa and Stephanie Rosenthal",
   editor = "IEEE",
   title = "Spatial references and perspective in natural language instructions for collaborative manipulation",
   booktitle = "IEEE International Symposium on Robot and Human Interactive Communication",
   month = "August",
   year = "2016",
}