Robust Depth Estimation using Auto-Exposure Bracketing

Sunghoon Im, Hae-Gon Jeon and In So Kweon
Journal Article, IEEE Transactions on Image Processing (TIP), December, 2018

View Publication

Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Abstract

As the computing power of hand-held devices grows, there has been increasing interest in the capture of depth information, to enable a variety of photographic applications. However, under low-light conditions, most devices still suffer from low imaging quality and inaccurate depth acquisition. To address the problem, we present a robust depth estimation method from a short burst shot with varied intensity (i.e., Auto-exposure bracketing) and/or strong noise (i.e., High ISO). Our key idea synergistically combines deep convolutional neural networks with geometric understanding of the scene. We introduce a geometric transformation between optical flow and depth tailored for burst images, enabling our learning-based multi-view stereo matching to be performed effectively. We then describe our depth estimation pipeline that incorporates this geometric transformation into our residual-flow network. It allows our framework to produce an accurate depth map even with a bracketed image sequence. We demonstrate that our method outperforms state-of-the-art methods for various datasets captured by a smartphone and a DSLR camera. Moreover, we show that the estimated depth is applicable for image quality enhancement and photographic editing.


@article{Im-2018-110410,
author = {Sunghoon Im and Hae-Gon Jeon and In So Kweon},
title = {Robust Depth Estimation using Auto-Exposure Bracketing},
journal = {IEEE Transactions on Image Processing (TIP)},
year = {2018},
month = {December},
keywords = {Depth estimation , exposure fusion , image denoising , 3D reconstruction , geometry , convolutional neural network},
} 2019-01-07T12:58:22-04:00