People with Visual Impairment Training Personal Object Recognizers: Feasibility and Challenges - Robotics Institute Carnegie Mellon University

People with Visual Impairment Training Personal Object Recognizers: Feasibility and Challenges

Hernisa Kacorri, Kris M. Kitani, Jeff Bigham, and Chieko Asakawa
Conference Paper, Proceedings of CHI Conference on Human Factors in Computing Systems (CHI '17), pp. 5839 - 5849, May, 2017

Abstract

Blind people often need to identify objects around them, from packages of food to items of clothing. Automatic object recognition continues to provide limited assistance in such tasks because models tend to be trained on images taken by sighted people with different background clutter, scale, viewpoints, occlusion, and image quality than in photos taken by blind users. We explore personal object recognizers, where visually impaired people train a mobile application with a few snapshots of objects of interest and provide custom labels. We adopt transfer learning with a deep learning system for user-defined multi-label k-instance classification. Experiments with blind participants demonstrate the feasibility of our approach, which reaches accuracies over 90% for some participants. We analyze user data and feedback to explore effects of sample size, photo-quality variance, and object shape; and contrast models trained on photos by blind participants to those by sighted participants and generic recognizers.

BibTeX

@conference{Kacorri-2017-109796,
author = {Hernisa Kacorri and Kris M. Kitani and Jeff Bigham and Chieko Asakawa},
title = {People with Visual Impairment Training Personal Object Recognizers: Feasibility and Challenges},
booktitle = {Proceedings of CHI Conference on Human Factors in Computing Systems (CHI '17)},
year = {2017},
month = {May},
pages = {5839 - 5849},
}