Shape-From-Silhouette of Articulated Objects and its Use for Human Body Kinematics Estimation and Motion Capture

Kong Man Cheung, Simon Baker, and Takeo Kanade
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, July, 2003.


Download
  • Adobe portable document format (pdf) (956KB)
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Abstract
Shape-From-Silhouette (SFS), also known as Visual Hull (VH) construction, is a popular 3D reconstruction method which estimates the shape of an object from multiple silhouette images. The original SFS formulation assumes that all of the silhouette images are captured either at the same time or while the object is static. This assumption is violated when the object moves or changes shape. Hence the use of SFS with moving objects has been restricted to treating each time instant sequentially and independently. Recently we have successfully extended the traditional SFS formulation to refine the shape of a rigidly moving object over time. Here we further extend SFS to apply to dynamic articulated objects. Given silhouettes of a moving articulated object, the process of recovering the shape and motion requires two steps: (1) correctly segmenting (points on the boundary of) the silhouettes to each articulated part of the object, (2) estimating the motion of each individual part using the segmented silhouette. In this paper, we propose an iterative algorithm to solve this simultaneous assignment and alignment problem. Once we have estimated the shape and motion of each part of the object, the articulation points between each pair of rigid parts are obtained by solving a simple motion constraint between the connected parts. To validate our algorithm, we first apply it to segment the different body parts and estimate the joint positions of a person. The acquired kinematic (shape and joint) information is then used to track the motion of the person in new video sequences.

Notes
Associated Center(s) / Consortia: Vision and Autonomous Systems Center
Associated Lab(s) / Group(s): Virtualized RealityTM
Associated Project(s): Temporal Shape-From-Silhouette and Human Kinematic Modeling and Motion Capture

Text Reference
Kong Man Cheung, Simon Baker, and Takeo Kanade, "Shape-From-Silhouette of Articulated Objects and its Use for Human Body Kinematics Estimation and Motion Capture," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, July, 2003.

BibTeX Reference
@inproceedings{Cheung_2003_4349,
   author = "Kong Man Cheung and Simon Baker and Takeo Kanade",
   title = "Shape-From-Silhouette of Articulated Objects and its Use for Human Body Kinematics Estimation and Motion Capture",
   booktitle = "Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition",
   month = "July",
   year = "2003",
}