/AAMs with Occlusion

AAMs with Occlusion

Portrait of AAMs with Occlusion
Heads: Ralph Gross and Simon Baker
Contact: Ralph Gross
Last Project Publication Year: 2006

Active Appearance Models (AAMs) are generative parametric models that have been successfully used in the past to track faces in video. A variety of video applications are possible, including dynamic head pose and gaze estimation for real-time user interfaces, lip-reading, and expression recognition. To construct an AAM, a number of training images of faces with a mesh of canonical feature points (usually hand-marked) are needed. All feature points have to be visible in all training images. However, in many scenarios parts of the face may be occluded. Perhaps the most common cause of occlusion is 3D pose variation, which can cause self-occlusion of the face. Furthermore, tracking using standard AAM fitting algorithms often fails in the presence of even small occlusions. In this project we developed algorithms to construct AAMs from occluded training images and to track faces efficiently in videos containing occlusion. We evaluated our algorithms both quantitatively and qualitatively and showed successful real-time face tracking on a number of image sequences containing varying degrees and types of occlusions.

The image sequences are shown below.


shape_modes.mpg
Shape modes generated from unoccluded and occluded training data. The resulting shape modes are very similar.

app_modes.mpg
Appearance modes generated from unoccluded and occluded training data. As shown in the paper the modes generated from unoccluded data and from data containing up to 45% of occlusion are very similar.


fit.mpg
Accurate tracking using an AAM constructed using image sequences containing occlusion.

box.mpg
Comparison of using the (non-robust) project-out algorithm and the efficient robust normalization algorithm on an image sequence with occlusion by a black box.


hand.mpg
Comparison of using the (non-robust) project-out algorithm and the efficient robust normalization algorithm on an image sequence with occlusion by a hand.

rotate.mpg
Comparison of using the (non-robust) project-out algorithm and the efficient robust normalization algorithm on an image sequence with self-occlusion.
Displaying 5 Publications
Evaluating Error Functions for Robust Active Appearance Models
Barry-John Theobald, Iain Matthews and Simon Baker

Conference Paper, Proceedings of the International Conference on Automatic Face and Gesture Recognition, pp. 149 - 154, April, 2006
Active Appearance Models with Occlusion
Ralph Gross, Iain Matthews and Simon Baker

Journal Article, Image and Vision Computing, Vol. 24, No. 6, pp. 593-604, January, 2006
Constructing and Fitting Active Appearance Models With Occlusion
Ralph Gross, Iain Matthews and Simon Baker

Conference Paper, Proceedings of the IEEE Workshop on Face Processing in Video, June, 2004
Lucas-Kanade 20 Years On: A Unifying Framework: Part 3
Simon Baker, Ralph Gross and Iain Matthews

Tech. Report, CMU-RI-TR-03-35, Robotics Institute, Carnegie Mellon University, November, 2003
Lucas-Kanade 20 Years On: A Unifying Framework: Part 2
Simon Baker, Ralph Gross, Iain Matthews and Takahiro Ishikawa

Tech. Report, CMU-RI-TR-03-01, Robotics Institute, Carnegie Mellon University, February, 2003

Current RI People

2017-09-13T10:44:46+00:00