Real-Time Combined 2D+3D Active Appearance Models

Jing Xiao, Simon Baker, Iain Matthews, and Takeo Kanade
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, June, 2004, pp. 535 - 542.


Download
  • Adobe portable document format (pdf) (5MB)
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Abstract
Active Appearance Models (AAMs) are generative models commonly used to model faces. Another closely related type of face models are 3D Morphable Models (3DMMs). Although AAMs are 2D, they can still be used to model 3D phenomena such as faces moving across pose. We first study the representational power of AAMs and show that they can model anything a 3DMM can, but possibly require more shape parameters. We quantify the number of additional parameters required and show that 2D AAMs can generate model instances that are not possible with the equivalent 3DMM. We proceed to describe how a non-rigid structure-from-motion algorithm can be used to construct the corresponding 3D shape modes of a 2D AAM. We then show how the 3D modes can be used to constrain the AAM so that it can only generate model instances that can also be generated with the 3D modes. Finally, we propose a real-time algorithm for fitting the AAM while enforcing the constraints, creating what we call a ``Combined 2D+3D AAM.''

Keywords
Active Appearance Models, 3D Morphable Models, Combined 2D+3D Active Appearance Models, Real-Time Fitting, Non-Rigid Face Tracking, Non-Rigid Structure-From-Motion

Notes
Sponsor: US Department of Defense
Grant ID: N41756-03-C4024
Associated Center(s) / Consortia: Vision and Autonomous Systems Center
Associated Lab(s) / Group(s): Vision for Safe Driving, People Image Analysis Consortium, Human Identification at a Distance, Face Group
Associated Project(s): Face Model Building and Fitting, AAM Fitting Algorithms, 2D->3D Face Model Construction
Number of pages: 8

Text Reference
Jing Xiao, Simon Baker, Iain Matthews, and Takeo Kanade, "Real-Time Combined 2D+3D Active Appearance Models," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, June, 2004, pp. 535 - 542.

BibTeX Reference
@inproceedings{Xiao_2004_4636,
   author = "Jing Xiao and Simon Baker and Iain Matthews and Takeo Kanade",
   title = "Real-Time Combined 2D+3D Active Appearance Models",
   booktitle = "Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition",
   pages = "535 - 542",
   month = "June",
   year = "2004",
   volume = "2",
}