/Recognizing Hand-Object Interactions in Wearable Camera Videos

Recognizing Hand-Object Interactions in Wearable Camera Videos

Tatsuya Ishihara, Kris M. Kitani, Wei-Chiu Ma, Hironobu Takagi and Chieko Asakawa
Conference Paper, IEEE International Conference on Image Processing (ICIP), September, 2015

Download Publication (PDF)

Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author’s copyright. These works may not be reposted without the explicit permission of the copyright holder.


Wearable computing technologies are advancing rapidly and enabling users to easily record daily activities for applications such as life-logging or health monitoring. Recognizing hand and object interactions in these videos will help broaden application domains, but recognizing such interactions automatically remains a difficult task. Activity recognition from the first-person point-of-view is difficult because the video includes constant motion, cluttered backgrounds, and sudden changes of scenery. Recognizing hand-related activities is particularly challenging due to the many temporal and spatial variations induced by hand interactions. We present a novel approach to recognize hand-object interactions by extracting both local motion features representing the subtle movements of the hands and global hand shape features to capture grasp types. We validate our approach on multiple egocentric action datasets and show that state-of-the-art performance can be achieved by considering both local motion and global appearance information.

BibTeX Reference
author = {Tatsuya Ishihara and Kris M. Kitani and Wei-Chiu Ma and Hironobu Takagi and and Chieko Asakawa},
title = {Recognizing Hand-Object Interactions in Wearable Camera Videos},
booktitle = {IEEE International Conference on Image Processing (ICIP)},
year = {2015},
month = {September},