3D Shape Completion and Canonical Pose Estimation with Structured Neural Networks

Abstract

3D point cloud is an efficient and flexible representation of 3D structures and the raw output of many 3D sensors. Recently, neural networks operating on point clouds have shown superior performance on various 3D understanding tasks, thanks to their power to extract task-specific semantic features directly from points. In this thesis, we study how to incorporate geometric and algorithmic structures into the design of neural networks in order to achieve more effective and efficient learning from raw point clouds. In particular, we investigate two problems – 3D shape completion and canonical pose estimation – that address two essential characteristics of 3D data in the wild: incompleteness and misalignment. We show that the structured neural networks we propose outperform alternative approaches that do not incorporate structural priors on synthetic benchmarks and demonstrate the potential of our networks to operate on challenging real world data such as LiDAR scans collected from an autonomous vehicle.