Interactive Multi-Modal Robot Programming

Soshi Iba, Chris Paredis, and Pradeep Khosla
Proceedings of the International Conf. on Robotics and Automations, May, 2002, pp. 161-168.

  • Adobe portable document format (pdf) (3MB)
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

As robots enter the human environment and come in contact with inexperienced users, they need to be able to interact with users in a multi-modal fashion?eyboard and mouse are no longer acceptable as the only input modalities.

This paper introduces a novel approach to program a robot interactively through a multi-modal interface. The key characteristic of this approach is that the user can provide feedback interactively at any time?uring both the programming and the execution phase. The framework takes a three-step approach to the problem: multi-modal recognition, intention interpretation, and prioritized task execution. The multi-modal recognition module translates hand gestures and spontaneous speech into a structured symbolic data stream without abstracting away the user? intent. The intention interpretation module selects the appropriate primitives to generate a task based on the user? input, the system? current state, and robot sensor data. Finally, the prioritized task execution module selects and executes skill primitives based on the system? current state, sensor inputs, and prior tasks. The framework is demonstrated by interactively controlling and programming a vacuum-cleaning robot.

Number of pages: 8

Text Reference
Soshi Iba, Chris Paredis, and Pradeep Khosla, "Interactive Multi-Modal Robot Programming," Proceedings of the International Conf. on Robotics and Automations, May, 2002, pp. 161-168.

BibTeX Reference
   author = "Soshi Iba and Chris Paredis and Pradeep Khosla",
   title = "Interactive Multi-Modal Robot Programming",
   booktitle = "Proceedings of the International Conf. on Robotics and Automations",
   pages = "161-168",
   month = "May",
   year = "2002",