Automatic eyeglasses removal from face images

Chenyu Wu, Ce Liu, Heung-Yueng Shum, Ying-Qing Xy, and Zhengyou Zhang
IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 26, No. 3, March, 2004, pp. 322 - 336.


Download
  • Adobe portable document format (pdf) (2MB)
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Abstract
In this paper, we present an intelligent image editing and face synthesis system that automatically removes eyeglasses from an input frontal face image. Although conventional image editing tools can be used to remove eyeglasses by pixel-level editing, filling in the deleted eyeglasses region with the right content is a difficult problem. Our approach works at the object level where the eyeglasses are automatically located, removed as one piece, and the void region filled. Our system consists of three parts: eyeglasses detection, eyeglasses localization, and eyeglasses removal. First, an eye region detector, trained offline, is used to approximately locate the region of eyes, thus the region of eyeglasses. A Markov-chain Monte Carlo method is then used to accurately locate key points on the eyeglasses frame by searching for the global optimum of the posterior. Subsequently, a novel sample-based approach is used to synthesize the face image without the eyeglasses. Specifically, we adopt a statistical analysis and synthesis approach to learn the mapping between pairs of face images with and without eyeglasses from a database. Extensive experiments demonstrate that our system effectively removes eyeglasses.

Notes
Number of pages: 15

Text Reference
Chenyu Wu, Ce Liu, Heung-Yueng Shum, Ying-Qing Xy, and Zhengyou Zhang, "Automatic eyeglasses removal from face images," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 26, No. 3, March, 2004, pp. 322 - 336.

BibTeX Reference
@article{Wu_2004_5517,
   author = "Chenyu Wu and Ce Liu and Heung-Yueng Shum and Ying-Qing Xy and Zhengyou Zhang",
   title = "Automatic eyeglasses removal from face images",
   journal = "IEEE Transactions on Pattern Analysis and Machine Intelligence",
   pages = "322 - 336",
   month = "March",
   year = "2004",
   volume = "26",
   number = "3",
}