A Lattice-based MRF model for Dynamic Near-regular Texture Tracking and Manipulation

Wen-Chieh Lin
doctoral dissertation, tech. report CMU-RI-TR-05-58, Robotics Institute, Carnegie Mellon University, December, 2005


Download
  • Adobe portable document format (pdf) (28MB)
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Abstract
A near-regular texture (NRT) is a geometric and photometric deformation from its regular origin ?a congruent wallpaper pattern formed by 2D translations of a single tile. A dynamic NRT is an NRT under motion. Correspondingly, the basic unit of a dynamic NRT is a well-defined texton, as a geometrically and photometrically deformed tile, moving through a 3D spatiotemporal space. Although NRTs are pervasive in man-made and natural environments, effective compu- tational algorithms for NRTs are few. Through a systematic and quantitative comparison study of multiple texture synthesis algorithms, we are able to show that faithful NRT synthesis has challenged most of the state of the art texture synthesis algorithms. Our recent work on static NRTs analysis and manipulation [Liu et al., 2004] is the first algorithmic treatment aimed specifically to preserve the regularity and randomness in real-world near regular textures. The theme of this thesis is to address computational issues in modeling, tracking and manipulating dynamic NRTs. One basic observation on dynamic NRT is its topology invariance property: the lattice structure of a dynamic NRT remains invariant despite its drastic geometry or appearance variations. We propose a lattice-based Markov- Random-Field (MRF) model for dynamic NRT in a 3D spatiotemporal space. Our dynamic NRT model consists of a global lattice structure that characterizes the topological constraint among multiple textons and an image observation model that handles local geometry and appearance variations. Our model behaves like a network of statistically varied springs. Based on our dynamic NRT model, we develop a tracking algorithm that can effectively handle the special challenges of dynamic NRT tracking, including: ambiguous correspondences, occlusions, illumination variations, and appearance variations. Our algorithm does not assume the type of motion that a dynamic NRT may undergo. Furthermore, we implement a dynamic NRT manipulation system that can replace and superimpose images on a dynamic NRT from an unknown environment. The main contributions of this thesis are: First, a novel and general quadrilateral lattice-based MRF model is proposed for dynamic NRT. Second, we implement a dynamic NRT tracking algorithm that can effectively handle real-world dynamic NRT with occlusions. Third, the proposed dynamic NRT framework makes it possible to accomplish several video editing and manipulation tasks, including real-world dynamic NRT synthesis, replacement, and superimposition

Notes
Associated Center(s) / Consortia: Vision and Autonomous Systems Center
Associated Lab(s) / Group(s): Computational Symmetry
Associated Project(s): Near Regular Texture -- Analysis, Synthesis and Manipulation
Number of pages: 160

Text Reference
Wen-Chieh Lin, "A Lattice-based MRF model for Dynamic Near-regular Texture Tracking and Manipulation," doctoral dissertation, tech. report CMU-RI-TR-05-58, Robotics Institute, Carnegie Mellon University, December, 2005

BibTeX Reference
@phdthesis{Lin_2005_5555,
   author = "Wen-Chieh Lin",
   title = "A Lattice-based MRF model for Dynamic Near-regular Texture Tracking and Manipulation",
   booktitle = "",
   school = "Robotics Institute, Carnegie Mellon University",
   month = "December",
   year = "2005",
   number= "CMU-RI-TR-05-58",
   address= "Pittsburgh, PA",
}