The Robotics Institute

Carnegie Mellon Robotics Institute

Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Abstract

We present an approach for modeling and rendering a dynamic, real-world event from an arbitrary viewpoint, and at any time, using images captured from multiple video cameras. The event is modeled as a non-rigidly varying dynamic scene, captured by many images from different viewpoints, at discrete times. First, the spatio-temporal geometric properties (shape and instantaneous motion) are computed. The view synthesis problem is then solved using a reverse mapping algorithm, ray-casting across space and time, to compute a novel image from any viewpoint in the 4D space of position and time. Results are shown on real-world events captured in the CMU 3D Room, by creating synthetic renderings of the event from novel, arbitrary positions in space and time. Multiple such re-created renderings can be put together to create re-timed fly-by movies of the event, with the resulting visual experience richer than that of a regular video clip, or switching between images from multiple cameras.

Keywords

Image-based modeling and rendering, dynamicscenes, spatio-temporal view interpolation, non-rigid motion, voxelmodels, space carving, scene flow.

Notes

Associated Center(s) / Consortia: Vision and Autonomous Systems Center

Associated Lab(s) / Group(s): Vision for Virtual Environments and Virtualized Reality^TM

Associated Project(s): Spatio-Temporal View Interpolation and Scene Flow

Number of pages: 32

Text Reference

Sundar Vedula, Simon Baker, and Takeo Kanade, "Image-Based Spatio-Temporal Modeling and View Interpolation of Dynamic Events," ACM Transactions on Graphics, Vol. 24, No. 2, April, 2005, pp. 240 - 261.

BibTeX Reference

@article{Vedula_2005_4877,
   author = "Sundar Vedula and Simon Baker and Takeo Kanade",
   title = "Image-Based Spatio-Temporal Modeling and View Interpolation of Dynamic Events",
   journal = "ACM Transactions on Graphics",
   pages = "240 - 261",
   month = "April",
   year = "2005",
   volume = "24",
   number = "2",
}

The Robotics Institute

Menu