Carnegie Mellon University
Pixel-level Hand Detection in Ego-Centric Videos

Kris M. Kitani and Cheng Li
Conference on Computer Vision and Pattern Recognition, June, 2013.

  • Adobe portable document format (pdf) (4MB)
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

We address the task of pixel-level hand detection in the context of ego-centric cameras. Extracting hand regions in ego-centric videos is a critical step for understanding hand- object manipulation and analyzing hand-eye coordination. However, in contrast to traditional applications of hand de- tection, such as gesture interfaces or sign-language recog- nition, ego-centric videos present new challenges such as rapid changes in illuminations, significant camera motion and complex hand-object manipulations. To quantify the challenges and performance in this new domain, we present a fully labeled indoor/outdoor ego-centric hand detection benchmark dataset containing over 200 million labeled pix- els, which contains hand images taken under various illu- mination conditions. Using both our dataset and a pub- licly available ego-centric indoors dataset, we give exten- sive analysis of detection performance using a wide range of local appearance features. Our analysis highlights the effectiveness of sparse features and the importance of mod- eling global illumination. We propose a modeling strategy based on our findings and show that our model outperforms several baseline approaches.

First-person vision, hand detection

Sponsor: NSF QoLT
Associated Center(s) / Consortia: Quality of Life Technology Center

Text Reference
Kris M. Kitani and Cheng Li, "Pixel-level Hand Detection in Ego-Centric Videos," Conference on Computer Vision and Pattern Recognition, June, 2013.

BibTeX Reference
   author = "Kris M Kitani and Cheng Li",
   title = "Pixel-level Hand Detection in Ego-Centric Videos",
   booktitle = "Conference on Computer Vision and Pattern Recognition",
   month = "June",
   year = "2013",