Robust Stereo Matching with Surface Normal Prediction

Shuangli Zhang, Weijian Xie, Guofeng Zhang, Hujun Bao and Michael Kaess
Conference Paper, IEEE Intl. Conf. on Robotics and Automation, ICRA, May, 2017

Download Publication

Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.


Traditional stereo matching approaches generally have problems in handling textureless regions, strong occlusions and reflective regions that do not satisfy a Lambertian surface assumption. In this paper, we propose to combine the predicted surface normal by deep learning to overcome these inherent difficulties in stereo matching. With the selected reliable disparities from stereo matching method and effective edge fusion strategy, we can faithfully convert the predicted surface normal map to a disparity map by solving a least squares system which maintains discontinuity on object boundaries and continuity on other regions. Then we refine the disparity map iteratively by bilateral filtering-based completion and edge feature refinement. Experimental results on the Middlebury dataset and our own captured stereo sequences demonstrate the effectiveness of the proposed approach.

author = {Shuangli Zhang and Weijian Xie and Guofeng Zhang and Hujun Bao and Michael Kaess},
title = {Robust Stereo Matching with Surface Normal Prediction},
booktitle = {IEEE Intl. Conf. on Robotics and Automation, ICRA},
year = {2017},
month = {May},
} 2017-09-13T10:38:04-04:00