/People with Visual Impairment Training Personal Object Recognizers: Feasibility and Challenges

People with Visual Impairment Training Personal Object Recognizers: Feasibility and Challenges

Hernisa Kacorri, Kris M. Kitani, Jeff Bigham and Chieko Asakawa
Conference Paper, Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17), pp. 5839 - 5849, May, 2017

Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author’s copyright. These works may not be reposted without the explicit permission of the copyright holder.

Abstract

Blind people often need to identify objects around them, from packages of food to items of clothing. Automatic object recognition continues to provide limited assistance in such tasks because models tend to be trained on images taken by sighted people with different background clutter, scale, viewpoints, occlusion, and image quality than in photos taken by blind users. We explore personal object recognizers, where visually impaired people train a mobile application with a few snapshots of objects of interest and provide custom labels. We adopt transfer learning with a deep learning system for user-defined multi-label k-instance classification. Experiments with blind participants demonstrate the feasibility of our approach, which reaches accuracies over 90% for some participants. We analyze user data and feedback to explore effects of sample size, photo-quality variance, and object shape; and contrast models trained on photos by blind participants to those by sighted participants and generic recognizers.

BibTeX Reference
@conference{Kacorri-2017-109796,
author = {Hernisa Kacorri and Kris M. Kitani and Jeff Bigham and Chieko Asakawa},
title = {People with Visual Impairment Training Personal Object Recognizers: Feasibility and Challenges},
booktitle = {Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17)},
year = {2017},
month = {May},
pages = {5839 - 5849},
}
2018-11-07T16:03:42+00:00