Camera-based Semantic Enhanced Vehicle Segmentation for Planar LIDAR

Chen Fu, Peiyun Hu, Chiyu Dong, Christoph Mertz and John M. Dolan
Conference Paper, Proceedings of the 2018 IEEE International Conference on Intelligent Transportation Systems, pp. 3805-3810, November, 2018

Download Publication

Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.


Vehicle segmentation is an important step in perception for autonomous driving vehicles, providing object-level environmental understanding. Its performance directly affects other functions in the autonomous driving car, including Decision-Making and Trajectory Planning. However, this task is challenging for planar LIDAR due to its limited vertical field of view (FOV) and quality of points. In addition, directly estimating 3D location, dimensions and heading of vehicles from an image is difficult due to the limited depth information of a monocular camera. We propose a method that fuses a vision-based instance segmentation algorithm and LIDAR-based segmentation algorithm to achieve an accurate 2D bird’s-eye view object segmentation. This method combines the advantages of both camera and LIDAR sensor: the camera helps to prevent over-segmentation in LIDAR, and LIDAR segmentation removes false positive areas in the interest regions in the vision results. A modified T-linkage RANSAC is applied to further remove outliers. A better segmentation also results in a better orientation estimation. We achieved a promising improvement in average absolute heading error and 2D IOU on both a reduced-resolution KITTI dataset and our Cadillac SRX planar LIDAR dataset.

author = {Chen Fu and Peiyun Hu and Chiyu Dong and Christoph Mertz and John M. Dolan},
title = {Camera-based Semantic Enhanced Vehicle Segmentation for Planar LIDAR},
booktitle = {Proceedings of the 2018 IEEE International Conference on Intelligent Transportation Systems},
year = {2018},
month = {November},
pages = {3805-3810},
keywords = {autonomous driving, perception, object detection, segmentation, LIDAR},
} 2019-06-03T08:25:38-04:00