Occlusion Reasoning for Skeleton Extraction of Self-Occluded Tree Canopies

Abstract

In this work, we present a method to extract the skeleton of a self-occluded tree canopy by estimating the unobserved structures of the tree. A tree skeleton compactly describes the topological structure and contains useful information such as branch geometry, positions and hierarchy. This can be critical to planning contact interactions for agricultural manipulation, yet is difficult to gain due to occlusion by leaves, fruits and other branches. Our method uses an instance segmentation network to detect visible trunk, branches, and twigs. Then, based on the observed tree structures, we build a custom 3D likelihood map in the form of an occupancy grid to hypothesize on the presence of occluded skeletons through a series of minimum cost path searches. We show that our method outperforms baseline methods in highly occluded scenes, demonstrated through a set of experiments on a synthetic tree dataset. Qualitative results are also presented on a real tree dataset collected from the field.