The Robotics Institute

Carnegie Mellon Robotics Institute

Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Abstract

We propose an architecture for a real-time multimodal system, which provides non-contact, adaptive user interfacing for Computer-Assisted Surgery (CAS). The system, called M/ORIS (for Medical/Operating Room Interaction System) combines gesture interpretation as an explicit interaction modality with continuous, real-time monitoring of the surgical activity in order to automatically address the surgeon's needs. Such a system will help reduce a surgeon's workload and operation time. This paper focuses on the proposed activity monitoring aspect of M/ORIS. We analyze the issues of Human-Computer Interaction in an OR based on real-world case studies. We then describe how we intend to address these issues by combining a surgical procedure description with parameters gathered from vision-based surgeon tracking and other OR sensors (e.g. tool trackers). We call this approach Scenario-based Procedure and Activity Monitoring (SPAM). We finally present preliminary results, including a non-contact mouse interface for surgical navigation systems.

Keywords

human-computer interaction, computer vision, computer aided surgery

Notes

Text Reference

Sebastien Grange, Terrence W. Fong, and Charles Baur, "M/ORIS: A medical/operating room interaction system," International Conference on Multimodal Interfaces (ICMI), October, 2004.

BibTeX Reference

@inproceedings{Fong_2004_4739,
   author = "Sebastien Grange and Terrence W Fong and Charles Baur",
   title = "M/ORIS: A medical/operating room interaction system",
   booktitle = "International Conference on Multimodal Interfaces (ICMI)",
   publisher = "ACM",
   month = "October",
   year = "2004",
}

The Robotics Institute

Menu