The Robotics Institute

Carnegie Mellon Robotics Institute

Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Abstract

Humans have an amazing ability to instantly grasp the overall 3D structure of a scene?round orientation, relative positions of major landmarks, etc.?ven from a single image. This ability is completely missing in most popular recognition algorithms, which pretend that the world is flat and/or view it through a patch-sized peephole. Yet it seems very likely that having a grasp of this ?urface layout?of a scene should be of great assistance for many tasks, including recognition, navigation, and novel view synthesis.

In this paper, we take the first step towards constructing the surface layout, a labeling of the image intogeometric classes. Our main insight is to learn appearance-based models of these geometric classes, which coarsely describe the 3D scene orientation of each image region. Our multiple segmentation framework provides robust spatial support, allowing a wide variety of cues (e.g., color, texture, and perspective) to contribute to the confidence in each geometric label. In experiments on a large set of outdoor images, we evaluate the impact of the individual cues and design choices in our algorithm. We further demonstrate the applicability of our method to indoor images, describe potential applications, and discuss extensions to a more complete notion of surface layout.

Keywords

surface layout, spatial layout, geometric context, scene understanding, context, object detection, model-driven segmentation, image understanding, multiple segmentations, object recognition

Notes

Sponsor: NSF

Associated Project(s): Geometrically Coherent Image Interpretation

Number of pages: 22

Text Reference

Derek Hoiem, Alexei A. Efros, and Martial Hebert, "Recovering Surface Layout from an Image," International Journal of Computer Vision, Vol. 75, No. 1, October, 2007

BibTeX Reference

@article{Hoiem_2007_5818,
   author = "Derek Hoiem and Alexei A. Efros and Martial Hebert",
   title = "Recovering Surface Layout from an Image",
   journal = "International Journal of Computer Vision",
   month = "October",
   year = "2007",
   volume = "75",
   number = "1",
}

The Robotics Institute

Menu