High-Zoom Video Hallucination by Exploiting Spatio-Temporal Regularities

Goksel Dedeoglu, Takeo Kanade, and Jonas August
Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '04), June, 2004, pp. 151 - 158.


Download
  • Adobe portable document format (pdf) (503KB)
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Abstract
In this paper, we consider the problem of super-resolving a human face video by a very high (x16) zoom factor. Inspired by recent literature on hallucination and example-based learning, we formulate this task using a graphical model that encodes 1) spatio-temporal consistencies, and 2) image formation & degradation processes. A video database of facial expressions is used to learn a domain-specific prior for high-resolution videos. The problem is posed as one of probabilistic inference, in which we aim to find the high resolution video that best satisfies the constraints expressed through the graphical model. Traditional approaches to this problem using video data first estimate the relative motion between frames and then compensate for it, effectively resulting in multiple measurements of the scene. Our use of time is rather direct: We define data structures that span multiple consecutive frames, enriching our feature vectors with a temporal signature. We then exploit these signatures to find consistent solutions over time. In our experiments, a 8x6 pixel-wide face video, subject to translational jitter and additive noise, gets magnified to a 128x96 pixel video. Our results show that by exploiting both space and time, drastic improvements can be achieved in both video flicker artifacts and mean-squared-error.

Keywords
super-resolution, face enhancement, hallucination

Notes
Associated Center(s) / Consortia: Vision and Autonomous Systems Center
Associated Lab(s) / Group(s): Face Group
Associated Project(s): Image Enhancement for Faces and Face Video Hallucination
Number of pages: 8

Text Reference
Goksel Dedeoglu, Takeo Kanade, and Jonas August, "High-Zoom Video Hallucination by Exploiting Spatio-Temporal Regularities," Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '04), June, 2004, pp. 151 - 158.

BibTeX Reference
@inproceedings{Dedeoglu_2004_4639,
   author = "Goksel Dedeoglu and Takeo Kanade and Jonas August",
   title = "High-Zoom Video Hallucination by Exploiting Spatio-Temporal Regularities",
   booktitle = "Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '04)",
   pages = "151 - 158",
   month = "June",
   year = "2004",
   volume = "2",
}