Multi-view Spatial and Temporal Interpolation for Dynamic Event Visualization

Abstract

The problem of visualizing dynamic real-world events on a digital computer has been a challenging one so far. We present a framework and algorithm to model and visualize these events by creating virtual images from synthetic cameras located at arbitrary viewpoints, at arbitrary time instants. We discuss Geometry, Radiance, and Motion Map (GRAMM) as an image based representation of shape, texture, and instantaneous motion of a scene. A GRAMM allows us to create virtual images, by interpolating between sets of images taken at finitely sampled positions in location and in time. This interpolation requires algorithms for estimation of 3-D scene motion, image based interpolation, and imposition of multi-camera geometric constraints. Applications include visualization of complex surgical procedures, entertainment, or training, where the physical locations and/or image capture rates of cameras are constrained, but there is a need to accurately reconstruct the appearance from any position, at any time instant.