RI | Research | Labs | Virtualized RealityTM

Text only version of this site

Virtualized RealityTM Head: Takeo Kanade Contact: Iain Matthews (iainm@cs.cmu.edu (Currently On Leave)) Mailing address: Carnegie Mellon University Robotics Institute 5000 Forbes Avenue Pittsburgh, PA 15213 Associated center: VASC For more information, see this lab's homepage.

This page last updated - November 2006.

Lab Description

Have you ever wondered what it would be like to watch a football game from the 50-yard line? No, not in seats on the side of the field, but actually ON the field? Or how about watching a basketball game from center court, running with the players? Although the idea is great, it usually isn't wise to put your chair in the middle of the action like that! Since 1993, we have been developing a technology that would allow you to see these and even wilder views of the world!

Virtualized Reality technology points a set of cameras at an event, and allows the viewer to virtually fly around, and watch the event from completely new positions. In a football game, you could watch the game from the point of view of the quarterback! Read on for more technical details.

Latest Results

Flamenco Dance

Man + Chair

3-Man Basketball

Earlier Results

Still Older Results

Volleyball

Baseball

"One-on-One" Basketball

Meditation

The 3D Room

Datasets & Programs

Download Data Sets (meditation and baseball)

Sample Program for Digitizing Images Using the Imagenation PXC200

Acknowledgements

Partial support provided by Intel Corporation, K²T,Inc., Matsushita Electric Industrial Company, Sony Corporation, and Zaxel Systems, Inc.

Related projects in the CMU Vision and Autonomous Systems Center

• Video rate multibaseline stereo machine
• Intelligent Sensor, Measurement, and Control Lab
• Robot Spatial Perception in the Field Robotics Center

Work at other sites

Personnel [Past members]

	Name	Title	Email Address
	Takeo Kanade	U.A. and Helen Whitaker University Prof.	tk@cs.cmu.edu
	Iain Matthews	Senior Systems Scientist	iainm@cs.cmu.edu (Currently On Leave)

Current Projects [Past Projects]

EyeVision	Human Kinematic Modeling and Motion Capture - We are developing a system for building 3D kinematic models of humans and then using the models to track the person in new video sequences.
Human Motion Transfer - We are developing a system for capturing the motion of one person and rendering a different person performing the same motion.	Temporal Shape-From-Silhouette - We are developing algorithms for the computation of 3D shape from multiple silhouette images captured across time.

Recent publications [View all 41 publications]

• Shape-From-Silhouette Across Time: Part II: Applications to Human Modeling and Markerless Motion Tracking
  K.M. Cheung, S. Baker, and T. Kanade
  International Journal of Computer Vision, Vol. 63, No. 3, August, 2005, pp. 225 - 245.
  Download: pdf [2755 KB] copyrighted

• Shape-From-Silhouette Across Time Part I: Theory and Algorithms
  K.M. Cheung, S. Baker, and T. Kanade
  International Journal of Computer Vision, Vol. 62, No. 3, May, 2005, pp. 221 - 247. [Abstract]
  Download: pdf [2047 KB] copyrighted

• Image-Based Spatio-Temporal Modeling and View Interpolation of Dynamic Events
  S. Vedula, S. Baker, and T. Kanade
  ACM Transactions on Graphics, Vol. 24, No. 2, April, 2005, pp. 240 - 261. [Abstract]
  Download: pdf [2053 KB] copyrighted

• Three-Dimensional Scene Flow
  S. Vedula, S. Baker, P. Rander, R. Collins, and T. Kanade
  IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 27, No. 3, March, 2005, pp. 475 - 480. [Abstract]
  Download: pdf [576 KB] copyrighted

• Markerless Human Motion Transfer
  K.M. Cheung, S. Baker, J.K. Hodgins, and T. Kanade
  Proceedings of the 2nd International Symposium on 3D Data Processing, Visualization and Transmission, September, 2004. [Abstract]
  Download: pdf [1216 KB] copyrighted

• Markerless Human Motion Transfer
  K.M. Cheung, S. Baker, J.K. Hodgins, and T. Kanade
  In Proceedings of the SIGGRAPH 2004 Conference on Sketches & Applications, ACM Press, August, 2004.
  Download: pdf [423 KB] copyrighted

• The CMU Pose, Illumination, and Expression Database
  T. Sim, S. Baker, and M. Bsat
  IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 25, No. 12, December, 2003, pp. 1615 - 1618. [Abstract]
  Download: pdf [369 KB] copyrighted

• Visual Hull Construction, Alignment and Refinement for Human Kinematic Modeling, Motion Tracking and Rendering
  K.M. Cheung
  doctoral dissertation, tech. report CMU-RI-TR-03-44, Robotics Institute, Carnegie Mellon University, October, 2003. [Abstract]
  Download: pdf [8142 KB] copyrighted

• Multi-Subregion Based Probabilistic Approach Toward Pose-Invariant Face Recognition
  T. Kanade and A. Yamada
  Proceedings of 2003 IEEE International Symposium on Computational Intelligence in Robotics and Automation (CIRA), July, 2003, pp. 954 - 959. [Abstract]
  Download: pdf [352 KB] copyrighted

• Shape-From-Silhouette of Articulated Objects and its Use for Human Body Kinematics Estimation and Motion Capture
  K.M. Cheung, S. Baker, and T. Kanade
  Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, June, 2003. [Abstract]
  Download: pdf [956 KB] copyrighted