Activity Recognition for Physically-Embodied Agent Teams

Abstract

This thesis focuses on the problem of activity recognition for physically-embodied agent teams. We define team activity recognition as the process of identifying team behaviors from traces of agents?positions and orientations as they evolve over time; the goal is to completely annotate agent traces with: 1) the correct sequence of low-level actions performed by each agent; 2) an assignment of agents to teams and subteams; 3) the set of team plans consistent with the observed sequence. Activity traces are gathered from teams of humans or agents performing military tasks in urban environments. Team behavior annotations can be used for a wide variety of applications including virtual training environments, visual monitoring systems, and commentator agents. For many physical domains, coordinated team behaviors create distinctive spatio-temporal patterns that can be used to identify low-level action sequences; we demonstrate that this can be done in a way that is robust to spatial variations in the environment and human deviations during behavior execution. This thesis addresses the novel problem of agent-to-team assignment for team tasks where team composition, the mapping of agents into teams, changes over time; this allows the analysis of more complicated tasks in which agents must periodically divide into subteams. To do this, we introduce a new algorithm, Simultaneous Team Assignment and Behavior Recognition (STABR), that generates low-level action annotations from spatio-temporal agent traces. Finally, we extend methods in symbolic plan recognition to exploit both temporal constraints between behaviors and agent role constraints in team plans to reduce the number of state history hypotheses that our system must consider.