Virtual Postman – Real-Time Interactive Virtual Video

Abstract

This paper presents a new paradigm for data interaction called virtual video. The concept is that video streams can be interactively augmented in real-time by both addition and removal of visual information. Removed information must be seamlessly replaced with relevant ``background'' information. Added information can be in the form of computer graphics, as in conventional augmented reality, or imagery derived from the video stream itself. Adding video-derived imagery means that a ``real'' character can be simulated in different places or times in the video stream, or interacting (fictitiously) with other characters. Achieving this requires an understanding of the motion and appearance of the target character so that it can be realistically inserted. Video understanding technology is now sufficiently advanced to make interactive, real-time virtual video a possibility. An example is given in the form a computer game called Virtual Postman in which moving characters detected in a video stream can be ``killed'' and thus removed from that stream. Furthermore, characters detected in the video stream are stored, analysed for rigid or periodic motion and then smoothly re-inserted into the video stream at arbitrary places, times and scales as ``clones'' of live characters, or ``zombies'' of dead ones.