A Theory of Fermat Paths for Non-Line-of-Sight Shape Reconstruction

Shumian Xin, Sotiris Nousias, Kiriakos N. Kutulakos, Aswin C. Sankaranarayanan, Srinivasa G. Narasimhan and Ioannis Gkioulekas
Conference Paper, IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2019), June, 2019

Download Publication

Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.


We present a novel theory of Fermat paths of light between a known visible scene and an unknown object not in the line of sight of a transient camera. These light paths either obey specular reflection or are reflected by the object’s boundary, and hence encode the shape of the hidden object. We prove that Fermat paths correspond to discontinuities in the transient measurements. We then derive a novel constraint that relates the spatial derivatives of the path lengths at these discontinuities to the surface normal. Based on this theory, we present an algorithm, called Fermat Flow, to estimate the shape of the non-line-of-sight object. Our method allows, for the first time, accurate shape recovery of complex objects, ranging from diffuse to specular, that are hidden around the corner as well as hidden behind a diffuser. Finally, our approach is agnostic to the particular technology used for transient imaging. As such, we demonstrate mm-scale shape recovery from picosecond scale transients using a SPAD and ultrafast laser, as well as micron-scale reconstruction from femtosecond scale transients using interferometry. We believe our work is a significant advance over the state-of-the-art in non-line-of-sight imaging.

author = {Shumian Xin and Sotiris Nousias and Kiriakos N. Kutulakos and Aswin C. Sankaranarayanan and Srinivasa G. Narasimhan and Ioannis Gkioulekas},
title = {A Theory of Fermat Paths for Non-Line-of-Sight Shape Reconstruction},
booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2019)},
year = {2019},
month = {June},
} 2019-06-20T09:41:14-04:00