/Recovering Surface Layout from an Image

Recovering Surface Layout from an Image

Derek Hoiem, Alexei A. Efros and Martial Hebert
Journal Article, Carnegie Mellon University, International Journal of Computer Vision, Vol. 75, No. 1, October, 2007

Download Publication (PDF)

Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author’s copyright. These works may not be reposted without the explicit permission of the copyright holder.


Humans have an amazing ability to instantly grasp the overall 3D structure of a scene?round orientation, relative positions of major landmarks, etc.?ven from a single image. This ability is completely missing in most popular recognition algorithms, which pretend that the world is flat and/or view it through a patch-sized peephole. Yet it seems very likely that having a grasp of this ?urface layout?of a scene should be of great assistance for many tasks, including recognition, navigation, and novel view synthesis. In this paper, we take the first step towards constructing the surface layout, a labeling of the image intogeometric classes. Our main insight is to learn appearance-based models of these geometric classes, which coarsely describe the 3D scene orientation of each image region. Our multiple segmentation framework provides robust spatial support, allowing a wide variety of cues (e.g., color, texture, and perspective) to contribute to the confidence in each geometric label. In experiments on a large set of outdoor images, we evaluate the impact of the individual cues and design choices in our algorithm. We further demonstrate the applicability of our method to indoor images, describe potential applications, and discuss extensions to a more complete notion of surface layout.

BibTeX Reference
author = {Derek Hoiem and Alexei A. Efros and Martial Hebert},
title = {Recovering Surface Layout from an Image},
journal = {International Journal of Computer Vision},
year = {2007},
month = {October},
volume = {75},
number = {1},
keywords = {surface layout, spatial layout, geometric context, scene understanding, context, object detection, model-driven segmentation, image understanding, multiple segmentations, object recognition},