Acquiring hand-action models in task and behavior levels by a learning robot through observing human demonstrations

Abstract

This paper describes our current research on learning tasks (what to do) and behaviors (how to do it) by a robot through observation of human demonstrations. We focus on human hand actions and represent such hand actions in symbolic behavior and task models. We propose frameworks of such models, derived a method to acquire those models through observation, evaluated them by using a human-form robot, and considered a method to improve such models automatically. We approach the problems on two levels: task and behavior. At task level acquisition, which obtains what-todo information, we propose a method for constructing a human task model by attention point (AP) analysis, which can efficiently integrate multiple input from several observation devices distributed in space and time, and then construct an abstract task model. As behavior level acquisition, which obtains how-to-do information, we present a method for segmenting a human assembly behavior into primitive behavior models, referred to as sub-skills; this method consists of analyzing face-contact transitions of manipulated objects in a human demonstration. We also describe the structure of the robot that we developed for learning and performing such hand-actions, and present experimental demonstrations performed by the robot.