Overlaying Ultrasound Images on Direct Vision

Abstract

The objective of this technical advance is to permit in situ visualization of ultrasonographic images so that direct hand-eye coordination can be used during invasive procedures. A method is presented that merges the visual outer surface of a patient with a simultaneous ultrasonographic scan of the patient's interior. The method combines a flat-panel monitor with a half-silvered mirror such that the image on the monitor is reflected precisely at the proper location within the patient. The ultrasonographic image is superimposed in real time on the patient, merging with the operator's hands and any invasive tools in the field of view. Instead of looking away from the patient at an ultrasonographic monitor, the opera- tor sees through skin and underlying tissue as if it were translucent. Two working proto- types have been constructed, demonstrating independence of viewer location and requiring no special apparatus to be worn by the operator. This method could enable needles and scalpels to be manipulated with direct hand-eye coordination under ultrasonographic guid- ance. Invasive tools would be visible up to where they enter the skin, permitting natural visual extrapolation into the ultrasonographic slice. Biopsy needles would no longer be restricted to lie in the plane of the ultrasonographic scan but could instead intersect it. These advances could lead to increased safety, ease, and reliability in certain invasive procedures. Key words: tomographic reflection; ultrasonographically guided biopsy; image overlay; aug- mented reality; visualization; percutaneous.