Carnegie Mellon University
Stereo Matching with Reflections and Translucency

Yanghai Tsin, Sing Bing Kang, and Richard Szeliski
IEEE Computer Vision and Pattern Recognition (CVPR'03), June, 2003.

  • Adobe portable document format (pdf) (3MB)
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

In this paper, we address the stereo matching problem in the presence of reflections and translucency, where image formation can be modeled as the additive superposition of layers at different depth. The presence of such effects violates the Lambertian assumption underlying traditional stereo vision algorithms, making it impossible to recover component depths using direct color matching based methods. We develop several techniques to estimate both depths and colors of the component layers. Depth hypotheses are enumerated in pairs, one from each layer, in a nested plane sweep. For each pair of depth hypotheses, we compute a component-color-independent matching error per pixel, using a spatial-temporal-differencing technique. We then use graph cut optimization to solve for the depths of both layers. This is followed by an iterative color update algorithm whose convergence is proven in our paper. We show convincing results of depth and color estimates for both synthetic and real image sequences.

stereo matching, reflection, translucency


Text Reference
Yanghai Tsin, Sing Bing Kang, and Richard Szeliski, "Stereo Matching with Reflections and Translucency," IEEE Computer Vision and Pattern Recognition (CVPR'03), June, 2003.

BibTeX Reference
   author = "Yanghai Tsin and Sing Bing Kang and Richard Szeliski",
   title = "Stereo Matching with Reflections and Translucency",
   booktitle = "IEEE Computer Vision and Pattern Recognition (CVPR'03)",
   month = "June",
   year = "2003",