RI Homepage Carnegie Mellon Homepage RI Homepage

The Robotics Institute

Carnegie Mellon Robotics Institute

A PDA-based Sign Translator

Jing Zhang, Xilin Chen, Jie Yang, and
Proceedings of the 2002 International Conference on Multimodal Interfaces (ICMI '02), October, 2002.


Download
  • Adobe portable document format (pdf) (267KB)
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Abstract
In this paper, we propose an effective approach for a PDA-based sign system, and it presents user the sign translator. Its main functions include 3 parts: detection, recognition and translation. Automatic detection and recognition of text in natural scenes is a prerequisite for automatic sign translator. In order to make the system robust for text detection in various natural scenes, the detection approach efficiently embeds multi-resolution, adaptive search in a hierarchical framework with different emphases at each layer. We also introduce an intensity-based OCR method to recognize character in various fonts and lighting condition, where we employ Gabor transform to obtain local features, and LDA for selection and classification of features. The recognition rate is 92.4% for the testing set got from the natural sign. Sign is different from the normal used sentence. It is brief, with a lot of abbreviations and place nouns. We here only briefly introduce a rule-based place name translation. We have integrated all these functions in a PDA, which can capture sign image, auto segment and recognize the Chinese sign, and translate it into English.

Notes

Text Reference
Jing Zhang, Xilin Chen, Jie Yang, and , "A PDA-based Sign Translator," Proceedings of the 2002 International Conference on Multimodal Interfaces (ICMI '02), October, 2002.

BibTeX Reference
@inproceedings{Yang_2002_4320,
   author = "Jing Zhang and Xilin Chen and Jie Yang and ",
   title = "A PDA-based Sign Translator",
   booktitle = "Proceedings of the 2002 International Conference on Multimodal Interfaces (ICMI '02)",
   month = "October",
   year = "2002",
}

The Robotics Institute is part of the School of Computer Science, Carnegie Mellon University.
Contact Us | Update Instructions