Real-time expression cloning using active appearance models

B. Theobald, Iain Matthews, Jeffrey Cohn, and S. Boker
Proceedings of the ACM International Conference on Multimodal Interfaces (ICMI'07), 2007, pp. 134 - 139.


Download
  • Adobe portable document format (pdf) (2MB)
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Abstract
Active Appearance Models (AAMs) are generative parametric models commonly used to track, recognise and synthesise faces in images and video sequences. In this paper we describe a method for transferring dynamic facial gestures between subjects in real-time. The main advantages of our approach are that: 1) the mapping is computed automatically and does not require high-level semantic information describing facial expressions or visual speech gestures. 2) The mapping is simple and intuitive, allowing expressions to be transferred and rendered in real-time. 3) The mapped expression can be constrained to have the appearance of the target producing the expression, rather than the source expression imposed onto the target face. 4) Near-videorealistic talking faces for new subjects can be created without the cost of recording and processing a complete training corpus for each. Our system enables face-to-face interaction with an avatar driven by an AAM of an actual person in real-time and we show examples of arbitrary expressive speech frames cloned across different subjects.

Notes
Associated Center(s) / Consortia: Vision and Autonomous Systems Center
Associated Lab(s) / Group(s): Face Group and Component Analysis
Associated Project(s): Facial Expression Analysis
Number of pages: 6

Text Reference
B. Theobald, Iain Matthews, Jeffrey Cohn, and S. Boker, "Real-time expression cloning using active appearance models," Proceedings of the ACM International Conference on Multimodal Interfaces (ICMI'07), 2007, pp. 134 - 139.

BibTeX Reference
@inproceedings{Matthews_2007_6013,
   author = "B. Theobald and Iain Matthews and Jeffrey Cohn and S. Boker",
   title = "Real-time expression cloning using active appearance models",
   booktitle = "Proceedings of the ACM International Conference on Multimodal Interfaces (ICMI'07)",
   pages = "134 - 139",
   year = "2007",
}