Shape-From-Silhouette Across Time Part I: Theory and Algorithms

Kong Man Cheung, Simon Baker, and Takeo Kanade
International Journal of Computer Vision, Vol. 62, No. 3, May, 2005, pp. 221 - 247.


Download
  • Adobe portable document format (pdf) (2MB)
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Abstract
Shape-From-Silhouette (SFS) is a shape reconstruction method which constructs a 3D shape estimate of an object using silhouette images of the object. The output of a SFS algorithm is known as the Visual Hull (VH). Traditionally SFS is either performed on static objects, or separately at each time instant in the case of videos of moving objects. In this paper we develop a theory of performing SFS across time: estimating the shape of a dynamic object (with unknown motion) by combining all of the silhouette images of the object over time. We first introduce a one dimensional element called a Bounding Edge to represent the Visual Hull. We then show that aligning two Visual Hulls using just their silhouettes is in general ambiguous and derive the geometric constraints (in terms of Bounding Edges) that govern the alignment. To break the alignment ambiguity, we combine stereo information with silhouette information and derive a Temporal SFS algorithm which consists of two steps: (1) estimate the motion of the objects over time (Visual Hull Alignment) and (2) combine the silhouette information using the estimated motion (Visual Hull Refinement). The algorithm is first developed for rigid objects and then extended to articulated objects. In the Part II of this paper we apply our temporal SFS algorithm to two human-related applications: (1) the acquisition of detailed human kinematic models and (2) marker-less motion tracking.

Keywords
3D Reconstruction, Shape-From-Silhouette, Visual Hull, Across Time, Stereo, Temporal Alignment, Alignment Ambiguity, Visibility

Notes
Associated Center(s) / Consortia: Vision and Autonomous Systems Center
Associated Lab(s) / Group(s): Virtualized RealityTM
Associated Project(s): Temporal Shape-From-Silhouette, Human Motion Transfer, Human Kinematic Modeling and Motion Capture
Number of pages: 51

Text Reference
Kong Man Cheung, Simon Baker, and Takeo Kanade, "Shape-From-Silhouette Across Time Part I: Theory and Algorithms," International Journal of Computer Vision, Vol. 62, No. 3, May, 2005, pp. 221 - 247.

BibTeX Reference
@article{Cheung_2005_4702,
   author = "Kong Man Cheung and Simon Baker and Takeo Kanade",
   title = "Shape-From-Silhouette Across Time Part I: Theory and Algorithms",
   journal = "International Journal of Computer Vision",
   pages = "221 - 247",
   month = "May",
   year = "2005",
   volume = "62",
   number = "3",
}