The Robotics Institute

Carnegie Mellon Robotics Institute

Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Abstract

In facial expression analysis, two principle approaches to extract facial features are geometric feature-based methods and appearance-based methods such as Gabor filters. In this paper, we combine these approaches in a feature-based system to recognize Facial Action Coding System (FACS) action units (AUs) in a complex database. The geometric facial features (including mouth, eyes, brows, and cheeks) are extracted using multi-state facial component models. After extraction, these features are represented parametricly. The regional facial appearance patterns are captured using a set of multi-scale and multi-orientation Gabor wavelet filters at specific locations. For the upper face, we recognize 8 AUs and neutral expression. The database consists of 606 image sequences from 107 adults of European, African, and Asian ancestry. AUs occur both alone and in combinations. Average recognition rate is 87.6% by using geometric facial features alone, 32% by using regional appearance patterns alone, 89.6% by combining both features, and 92.7% after refinement. For the lower face, we recognize 13 AUs and neutral expression. The database consists of 514 image sequences from 180 adults of European, African, and Asian ancestry. AUs occur both alone and in combinations. Average recognition rate is 84.7% by using geometric facial features alone, 82% by combining both features, and 87.4% after refinement.

Keywords

Facial expression analysis,Action units,Neural network,Geometric features,Regional Appearance Patterns,Gabor wavelet

Notes

Sponsor: NIMH

Grant ID: R01MH51435

Associated Center(s) / Consortia: Vision and Autonomous Systems Center

Associated Lab(s) / Group(s): Face Group

Associated Project(s): Face Databases

Text Reference

Ying-Li Tian, Takeo Kanade, and Jeffrey Cohn, "Recognizing Facial Actions by Combining Geometric Features and Regional Appearance Patterns," tech. report CMU-RI-TR-01-01, Robotics Institute, Carnegie Mellon University, January, 2001

BibTeX Reference

@techreport{Tian_2001_3461,
   author = "Ying-Li Tian and Takeo Kanade and Jeffrey Cohn",
   title = "Recognizing Facial Actions by Combining Geometric Features and Regional Appearance Patterns",
   booktitle = "",
   institution = "Robotics Institute",
   month = "January",
   year = "2001",
   number= "CMU-RI-TR-01-01",
   address= "Pittsburgh, PA",
}

The Robotics Institute

Menu