PALM: Portable Sensor-Augmented Vision System for Large Scene Modeling

Abstract

We propose PALM - a Portable sensor-Augmented vision system for Large-scene Modeling. The system is for recovering large structures in arbitrary scenes from video streams taken by a sensor-augmented camera. Central to the solution method is the combined use of multiple constraints derived from GPS measurements, camera orientation sensor readings, and image features. The knowledge of camera orientation allows for a linear formulation of perspective ray constraints, which results in substantial improvement of computational efficiency. The overall scene is reconstructed by merging smaller shape segments. Shape merging errors are minimized using the concept of shape hierarchy, which is realized through a "landmarking" technique. The features of the system include its use of a small number of images and feature points, its portability, and its low-cost interface for synchronizing sensor measurements with the video stream. The synchronization is achieved by storing the sensor readings in the audio channel of the camcorder. We built a hardware interface to convert RS232 signals to analog audio signals, and designed a software algorithm to decode the digitized audio signals back to the original sensor readings. Example reconstruction of a football stadium and three large buildings are presented and these results are compared with the ground truth.