Recovery of Dynamic Scene Structure from Multiple Image Sequences

Abstract

Despite significant progress in automatic recovery of static scene structure from range images, little effort has been made toward extending these approaches to dynamic scenes. This disparity is in large part due to the lack of range sensors with the high sampling rates needed to accurately capture dynamic scenes. We have developed a system that overcomes this problem by exploiting video cameras, which easily capture images of dynamic scenes, and image-based stereo, which estimates scene structure based on correspondences among the images from different cameras. Our system uses a synchronized multi-camera recording system to capture live video of the scene and a software implementation of image-based stereo to compute range images off-line. By combining this system with multi-image fusion, we created a novel system for dynamic structure recovery, with many applications including telepresence, training, and entertainment. Development of this system has also revealed the potential use effusion as both a multi-view and multi-resolution integration process for stereo.