/Semantic 3D Occupancy Mapping through Efficient High Order CRFs

Semantic 3D Occupancy Mapping through Efficient High Order CRFs

Shichao Yang, Yulan Huang and Sebastian Scherer
Conference Paper, IEEE/RSJ International Conference on Robots and Systems, July, 2017

Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author’s copyright. These works may not be reposted without the explicit permission of the copyright holder.


Semantic 3D mapping can be used for many applications such as robot navigation and virtual interaction. In recent years, there has been great progress in semantic segmentation and geometric 3D mapping. However, it is still challenging to combine these two tasks for accurate and large-scale semantic mapping from images. In the paper, we propose an incremental and (near) real-time semantic mapping system. A 3D scrolling occupancy grid map is built to represent the world, which is memory and computationally efficient and bounded for large scale environments. We utilize the CNN segmentation as prior prediction and further optimize 3D grid labels through a novel CRF model. Superpixels are utilized to enforce smoothness and form robust P N high order potential. An efficient mean field inference is developed for the graph optimization. We evaluate our system on the KITTI dataset and improve the segmentation accuracy by 10% over existing systems.

BibTeX Reference
author = {Shichao Yang and Yulan Huang and Sebastian Scherer},
title = {Semantic 3D Occupancy Mapping through Efficient High Order CRFs},
booktitle = {IEEE/RSJ International Conference on Robots and Systems},
year = {2017},
month = {July},