Carnegie Mellon University
Activity Recognition for Physically-Embodied Agent Teams

Gita Sukthankar
tech. report CMU-RI-05-44, Robotics Institute, Carnegie Mellon University, October, 2005

  • Adobe portable document format (pdf) (1MB)
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

This thesis focuses on the problem of activity recognition for physically-embodied agent teams. We define team activity recognition as the process of identifying team behaviors from traces of agents?positions and orientations as they evolve over time; the goal is to completely annotate agent traces with: 1) the correct sequence of low-level actions performed by each agent; 2) an assignment of agents to teams and subteams; 3) the set of team plans consistent with the observed sequence. Activity traces are gathered from teams of humans or agents performing military tasks in urban environments. Team behavior annotations can be used for a wide variety of applications including virtual training environments, visual monitoring systems, and commentator agents.

For many physical domains, coordinated team behaviors create distinctive spatio-temporal patterns that can be used to identify low-level action sequences; we demonstrate that this can be done in a way that is robust to spatial variations in the environment and human deviations during behavior execution. This thesis addresses the novel problem of agent-to-team assignment for team tasks where team composition, the mapping of agents into teams, changes over time; this allows the analysis of more complicated tasks in which agents must periodically divide into subteams. To do this, we introduce a new algorithm, Simultaneous Team Assignment and Behavior Recognition (STABR), that generates low-level action annotations from spatio-temporal agent traces. Finally, we extend methods in symbolic plan recognition to exploit both temporal constraints between behaviors and agent role constraints in team plans to reduce the number of state history hypotheses that our system must consider.

multi-agent plan recognition, activity inferencing, teamwork, spatial reasoning

Associated Center(s) / Consortia: Center for Integrated Manfacturing Decision Systems
Associated Lab(s) / Group(s): Advanced Agent - Robotics Technology Lab
Associated Project(s): AFOSR PRET: Information Fusion for Command and Control: The Translation of Raw Data To Actionable Knowledge and Decision

Text Reference
Gita Sukthankar, "Activity Recognition for Physically-Embodied Agent Teams," tech. report CMU-RI-05-44, Robotics Institute, Carnegie Mellon University, October, 2005

BibTeX Reference
   author = "Gita Sukthankar",
   title = "Activity Recognition for Physically-Embodied Agent Teams",
   booktitle = "",
   institution = "Robotics Institute",
   month = "October",
   year = "2005",
   number= "CMU-RI-05-44",
   address= "Pittsburgh, PA",