Graphics enhanced version of this site

Video-Rate Z Keying: A New Method for Merging Images
T. Kanade, K. Oda, A. Yoshida, M. Tanaka, and H. Kano
tech. report CMU-RI-TR-95-38, Robotics Institute, Carnegie Mellon University, December, 1995.

Jump to: Download | Abstract | Notes | Text Reference | BibTeX Reference

Adobe portable document format (pdf) [475 KB]
Compressed postscript (ps.gz) [564 KB]

Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Video-rate z keying is a new image keying method for merging real and synthetic images n real time. In visual media communication and display, it is often necessary to merge video signals from a real camera and a synthetic video produced by computer graphics. A standard technique for such a purpose is chroma keying which is used, for example, in TV weather reports. Chroma-keying, however, simply puts real world objects in the foreground of the synthetic image, and cannot deal with situation where real and synthetic objects occlude each other.

The z key method we present merges real and virtual world images in a more flexible way. The z key uses pixel-by-pixel depth information in the form of a depth map as a switch. For each pixel, the z key switch compared the pixel depth values of two images, and routes the color value of the foreground image that is nearer to the camera for the merged output image. The result of this pixel-by-pixel switching is that real and virtual objects can occlude each other correctly depending on their geometrical relationships.

The critical capability for realizing such video-rate z keying is video-rate pixel-by-pixel depth mapping of a real scene. We have developed a video-rate stereo machine which can produce 200 x 200 depth images at video rate. With this machine, merging a real scene with a synthetic scene by means of z keying in real-time has been demonstrated; a real person walks around in a synthetic room with correct relationships with virtual objects in the room at the rate of 15 frames/sec.

Grant ID: DACA76-89-C-0014, DAAE07-90-C-R059

Associated center: VASC
Associated lab/group: Vision for Virtual Environments
Associated project: Z-Keying

Number of pages: 8

T. Kanade, K. Oda, A. Yoshida, M. Tanaka, and H. Kano, Video-Rate Z Keying: A New Method for Merging Images, tech. report CMU-RI-TR-95-38, Robotics Institute, Carnegie Mellon University, December, 1995.

@techreport{Kanade_1995_390,
   author = "Takeo Kanade and Kazou Oda and Atsushi Yoshida and Masaya Tanaka and Hiroshi Kano",
   title = "Video-Rate Z Keying: A New Method for Merging Images",
   institution = "Robotics Institute, Carnegie Mellon University",
   month = "December",
   year = "1995",
   number = "CMU-RI-TR-95-38",
   address = "Pittsburgh, PA"
}