/3DNN: Viewpoint Invariant 3D Geometry Matching for Scene Understanding

3DNN: Viewpoint Invariant 3D Geometry Matching for Scene Understanding

Scott Satkin and Martial Hebert
Conference Paper, International Conference on Computer Vision (ICCV). 2013., December, 2013

Download Publication (PDF)

Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author’s copyright. These works may not be reposted without the explicit permission of the copyright holder.


We present a new algorithm 3DNN (3D Nearest- Neighbor), which is capable of matching an image with 3D data, independently of the viewpoint from which the image was captured. By leveraging rich annotations associated with each image, our algorithm can automatically produce precise and detailed 3D models of a scene from a single im- age. Moreover, we can transfer information across images to accurately label and segment objects in a scene. The true benefit of 3DNN compared to a traditional 2D nearest-neighbor approach is that by generalizing across viewpoints, we free ourselves from the need to have training examples captured from all possible viewpoints. Thus, we are able to achieve comparable results using orders of mag- nitude less data, and recognize objects from never-before- seen viewpoints. In this work, we describe the 3DNN algo- rithm and rigorously evaluate its performance for the tasks of geometry estimation and object detection/segmentation. By decoupling the viewpoint and the geometry of an image, we develop a scene matching approach which is truly 100% viewpoint invariant, yielding state-of-the-art performance on challenging data.

BibTeX Reference
author = {Scott Satkin and Martial Hebert},
title = {3DNN: Viewpoint Invariant 3D Geometry Matching for Scene Understanding},
booktitle = {International Conference on Computer Vision (ICCV). 2013.},
year = {2013},
month = {December},