The CMU/Pitt Automated Facial Image Analysis System - Robotics Institute Carnegie Mellon University

The CMU/Pitt Automated Facial Image Analysis System

Conference Paper, Proceedings of 5th International Conference on Methods and Techniques in Behavioral Research: Symposium 1: Automatic facial expression analysis and synthesis (MB '05), pp. 27 - 30, August, 2005

Abstract

Both the configuration and the timing of facial actions are important in emotion expression and recognition. To investigate the timing and configuration of facial actions, the CMU/Pitt Automated Facial Image Analysis (AFA) System has been developed. The latest version of the system is based on Active Appearance Models (AAMs). These are generative, parametric models and consist of a shape component and an appearance component. The shape component of is a triangulated mesh that moves like a face undergoing both rigid motion (head pose variation) and non-rigid motion (expression) in response to changes in the parameters. The appearance component of the AAM is an image of the face, which itself can vary under the control of the parameters. As the parameters are varied, the appearance varies so as to model effects such as the emergence of furrows and wrinkles and the visibility of the teeth as the mouth opens. Two disadvantages of traditional AAMs are that they are 2D and rigid head motion and non-rigid facial motion are confounded in the shape model. To address these problems, we recently developed an extension to AAMs that augments the usual 2D shape model with a 3D shape model. This advance separates the 3D rigid motion of the head and 3D nonrigid facial expression into two disjoint sets of parameters and recovers the 3D shape of the face. This approach works well as long as out-of-plane head motion is small to moderate. As out-of-plane head motion becomes large, automatic recovery of 3D shape becomes increasingly difficult because of self-occlusion. To solve this problem, a single AAM is fitted to multiple images captured
simultaneously from synchronized cameras.

BibTeX

@conference{Kanade-2005-16976,
author = {Takeo Kanade and Jeffrey Cohn},
title = {The CMU/Pitt Automated Facial Image Analysis System},
booktitle = {Proceedings of 5th International Conference on Methods and Techniques in Behavioral Research: Symposium 1: Automatic facial expression analysis and synthesis (MB '05)},
year = {2005},
month = {August},
pages = {27 - 30},
address = {Wageningen, The Netherlands},
}