A Visual/Haptic Interface to Virtual Environment: WYSIWYF Display

Abstract

To build a VR training system for visuo-motor skills (i.e., hand-eye coordinated skills), a visual interface should be correctly registered to a haptic interface so that the visual sensation and the haptic sensation are consistent spatially and temporally. In this paper, we propose a method to realize correct visual/haptic registration. The proposed method provides correct visual/haptic registration using a vision-based object tracking technique and a video keying technique. The combination of the encountered type haptic interface approach with the motion command type impedance display algorithm makes it possible to realize two extreme cases: free motion and rigid constraint. The first prototype has been built and the proposed concept was demonstrated.