Spatio-Temporal View Interpolation

Sundar Vedula, Simon Baker, and Takeo Kanade
tech. report CMU-RI-TR-01-35, Robotics Institute, Carnegie Mellon University, September, 2001

  • Adobe portable document format (pdf) (2MB)
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

We propose an algorithm for creating novel views of a non-rigidly varying dynamic event by combining images captured from different positions, at different times. The algorithm operates by combining images captured across space and time to compute voxel models of the scene shape at each time instant, and dense 3D scene flow between the voxel models (the non-rigid motion of every point in the scene). To interpolate in time the voxel models are ``flowed'' using the appropriate scene flow and a smooth surface fit to the result. The novel image is then computed by ray-casting to the surface at the intermediate time, following the scene flow to the neighboring time instants, projecting into the input images at those times, and finally blending the results. We use the algorithm to create re-timed slow-motion fly-by movies of real-world events.

Image Based Rendering, View Synthesis, Scene Flow, 3D Modeling

Associated Center(s) / Consortia: Vision and Autonomous Systems Center
Associated Lab(s) / Group(s): Virtualized RealityTM
Associated Project(s): Spatio-Temporal View Interpolation

Text Reference
Sundar Vedula, Simon Baker, and Takeo Kanade, "Spatio-Temporal View Interpolation," tech. report CMU-RI-TR-01-35, Robotics Institute, Carnegie Mellon University, September, 2001

BibTeX Reference
   author = "Sundar Vedula and Simon Baker and Takeo Kanade",
   title = "Spatio-Temporal View Interpolation",
   booktitle = "",
   institution = "Robotics Institute",
   month = "September",
   year = "2001",
   number= "CMU-RI-TR-01-35",
   address= "Pittsburgh, PA",