Temporal Segmentation and Activity Classification from First-person Sensing

Ekaterina H. Spriggs, Fernando De la Torre Frade, and Martial Hebert
IEEE Workshop on Egocentric Vision, CVPR 2009, July, 2009.


Download
  • Adobe portable document format (pdf) (4MB)
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Abstract
Temporal segmentation of human motion into actions is central to the understanding and building of computational models of human motion and activity recognition. Several issues contribute to the challenge of temporal segmentation and classification of human motion. These include the large variability in the temporal scale and periodicity of human actions, the complexity of representing articulated motion, and the exponential nature of all possible movement combinations. We provide initial results from investigating two distinct problems - classification of the overall task being performed, and the more difficult problem of classifying individual frames over time into specific actions. We explore first-person sensing through a wearable camera and Inertial Measurement Units (IMUs) for temporally segmenting human motion into actions and performing activity classification in the context of cooking and recipe preparation in a natural environment. We present baseline results for supervised and unsupervised temporal segmentation, and recipe recognition in the CMU-Multimodal activity database (CMU-MMAC).

Keywords
first-person vision, activity recognition, multi-modal data, CMU-MMAC

Notes
Sponsor: National Science Foundation
Associated Center(s) / Consortia: Vision and Autonomous Systems Center and Quality of Life Technology Center
Associated Lab(s) / Group(s): Vision and Mobile Robotics Lab, Human-Robot Interaction Group, Human Sensing
Associated Project(s): Quality of Life Technology

Text Reference
Ekaterina H. Spriggs, Fernando De la Torre Frade, and Martial Hebert, "Temporal Segmentation and Activity Classification from First-person Sensing," IEEE Workshop on Egocentric Vision, CVPR 2009, July, 2009.

BibTeX Reference
@inproceedings{De_la_Torre_Frade_2009_6394,
   author = "Ekaterina H. Spriggs and Fernando {De la Torre Frade} and Martial Hebert",
   title = "Temporal Segmentation and Activity Classification from First-person Sensing",
   booktitle = "IEEE Workshop on Egocentric Vision, CVPR 2009",
   month = "July",
   year = "2009",
}