Detection, tracking, and classification of subtle changes in facial expression

Jenn-Jier James Lien, Takeo Kanade, Jeffrey Cohn, and C. Li
Journal of Robotics and Autonomous Systems, Vol. 31, 2000, pp. 131 - 146.


Download
  • Adobe portable document format (pdf) (548KB)
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Abstract
Most of the current work on automated facial expression analysis attempt to recognize a small set of prototypic expressions, such as joy and fear. Such prototypic expressions, however, occur infrequently, and human emotions and intentions are communicated more often by changes in one or two discrete features. To capture the full range of facial expression, detection, tracking, and classification of fine-grained changes in facial features are needed. We developed the first version of a computer vision system that is sensitive to subtle changes in the face. The system includes three modules to extract feature information: dense-flow extraction using a wavelet motion model, facial feature tracking, and edge and line extraction. The feature information thus extracted is fed to discriminant classifiers or hidden Markov models that classify it into FACS action units, the descriptive system to code fine-grained changes in facial expression. The system was tested on image sequences from 100 male and female subjects of varied ethnicity. Agreement with manual FACS coding was strong for the results based on dense-flow extraction and facial feature tracking, and strong to moderate for edge and line extraction.

Notes
Associated Center(s) / Consortia: Vision and Autonomous Systems Center
Associated Lab(s) / Group(s): Face Group and Component Analysis
Associated Project(s): Facial Expression Analysis

Text Reference
Jenn-Jier James Lien, Takeo Kanade, Jeffrey Cohn, and C. Li, "Detection, tracking, and classification of subtle changes in facial expression," Journal of Robotics and Autonomous Systems, Vol. 31, 2000, pp. 131 - 146.

BibTeX Reference
@article{Lien_2000_4011,
   author = "Jenn-Jier James Lien and Takeo Kanade and Jeffrey Cohn and C. Li",
   title = "Detection, tracking, and classification of subtle changes in facial expression",
   journal = "Journal of Robotics and Autonomous Systems",
   pages = "131 - 146",
   year = "2000",
   volume = "31",
}