Modelling Human Assembly Actions from Observation

George Paul, Yunde Jiar, Mark D. Wheeler, and Katsushi Ikeuchi
IEEE/SICE/RSJ International Conference on Multisensor Fusion and Integration for Intelligent Systems, December, 1996.


Download
  • Adobe portable document format (pdf) (366KB)
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Abstract
This paper describes a system which can model an assembly task performed by a human. The actions are recorded in real-time using a stereo system. The assembled objects and the fingers of the hand are tracked through the image sequence. We use the spatial relations between the fingers and the objects to temporally segment the task into approach, pre-manipulate, manipulate and depart phases. We interpret the actions in each segment broadly into grasp, push, fine-motion etc. We then analyze the contact relations between objects during the manipulate phase to reconstruct the fine motion path of the manipulated object. The fine motion in configuration space is a series of connected path segments lying on the features (c-surfaces) of the configuration space obstacle. We project the observed configurations onto these c-surfaces and reconstruct the path segments. The connected path segments form the fine motion path. We demonstrate the system using the peg in hole task.

Notes
Associated Center(s) / Consortia: Vision and Autonomous Systems Center

Text Reference
George Paul, Yunde Jiar, Mark D. Wheeler, and Katsushi Ikeuchi, "Modelling Human Assembly Actions from Observation," IEEE/SICE/RSJ International Conference on Multisensor Fusion and Integration for Intelligent Systems, December, 1996.

BibTeX Reference
@inproceedings{Paul_1996_1704,
   author = "George Paul and Yunde Jiar and Mark D. Wheeler and Katsushi Ikeuchi",
   title = "Modelling Human Assembly Actions from Observation",
   booktitle = "IEEE/SICE/RSJ International Conference on Multisensor Fusion and Integration for Intelligent Systems",
   month = "December",
   year = "1996",
}