Video-Rate Z Keying: A New Method for Merging Images

Takeo Kanade, Kazou Oda, Atsushi Yoshida, Masaya Tanaka, and Hiroshi Kano
tech. report CMU-RI-TR-95-38, Robotics Institute, Carnegie Mellon University, December, 1995


Download
  • Adobe portable document format (pdf) (475KB)
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Abstract
Video-rate z keying is a new image keying method for merging real and synthetic images n real time. In visual media communication and display, it is often necessary to merge video signals from a real camera and a synthetic video produced by computer graphics. A standard technique for such a purpose is chroma keying which is used, for example, in TV weather reports. Chroma-keying, however, simply puts real world objects in the foreground of the synthetic image, and cannot deal with situation where real and synthetic objects occlude each other.

The z key method we present merges real and virtual world images in a more flexible way. The z key uses pixel-by-pixel depth information in the form of a depth map as a switch. For each pixel, the z key switch compared the pixel depth values of two images, and routes the color value of the foreground image that is nearer to the camera for the merged output image. The result of this pixel-by-pixel switching is that real and virtual objects can occlude each other correctly depending on their geometrical relationships.

The critical capability for realizing such video-rate z keying is video-rate pixel-by-pixel depth mapping of a real scene. We have developed a video-rate stereo machine which can produce 200 x 200 depth images at video rate. With this machine, merging a real scene with a synthetic scene by means of z keying in real-time has been demonstrated; a real person walks around in a synthetic room with correct relationships with virtual objects in the room at the rate of 15 frames/sec.


Notes
Grant ID: DACA76-89-C-0014, DAAE07-90-C-R059
Associated Center(s) / Consortia: Vision and Autonomous Systems Center
Associated Lab(s) / Group(s): Vision for Virtual Environments
Associated Project(s): Z-Keying
Number of pages: 8

Text Reference
Takeo Kanade, Kazou Oda, Atsushi Yoshida, Masaya Tanaka, and Hiroshi Kano, "Video-Rate Z Keying: A New Method for Merging Images," tech. report CMU-RI-TR-95-38, Robotics Institute, Carnegie Mellon University, December, 1995

BibTeX Reference
@techreport{Kanade_1995_390,
   author = "Takeo Kanade and Kazou Oda and Atsushi Yoshida and Masaya Tanaka and Hiroshi Kano",
   title = "Video-Rate Z Keying: A New Method for Merging Images",
   booktitle = "",
   institution = "Robotics Institute",
   month = "December",
   year = "1995",
   number= "CMU-RI-TR-95-38",
   address= "Pittsburgh, PA",
}