Inter and Intra Image Relationships in 3D Space

Nilesh Kulkarni
Master's Thesis, Tech. Report, CMU-RI-TR-19-31, August, 2019

Download Publication

Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Abstract

Relationships are at the core of understanding images. A lot of progress in computer vision is due to the ability of convolutional neural networks to model relationships between pixels in images across our massive data-sets. Convolutional neural networks bake in relationships between pixels both within images and across image collections to perform better on tasks like image segmentation, object detection, etc.. Relationship can exists in the pixel, concept and object space which is a huge spectrum. We show that different kinds of relationships are relevant for different tasks. This thesis explores various other ways to model “inter and intra image relationships” to help with the tasks of semantic correspondences, and 3D reconstruction.

In this thesis we explore the following two tasks a) semantic correspondence between images from the same category of objects where we show how relationships can be inherently learned and how geometric constraints helps us learn global meaningful relationships and b) 3D reconstruction from a single image where we explicitly factor relationships in 3D to help with the task of 3D reconstruction to improve the overall performance across multiple datasets.


@mastersthesis{Kulkarni-2019-117112,
author = {Nilesh Kulkarni},
title = {Inter and Intra Image Relationships in 3D Space},
year = {2019},
month = {August},
school = {},
address = {Pittsburgh, PA},
number = {CMU-RI-TR-19-31},
keywords = {3D, correspondence, single image, relationships},
} 2019-08-12T08:57:29-04:00