Abstract:
Anticipating the near-future actions of multiple people is central to embodied systems that plan and coordinate in shared environments, yet most research targets a single agent and ignores the inter-agent dependencies that shape group behavior. This thesis presents InteractFormer, a unified model that treats inter-agent interaction as a first-class signal: operating directly on fine-grained visual tokens, it lets agents attend to one another spatially and temporally to jointly predict all their futures. On two benchmarks—LEMMA (household collaboration) and SportsHHI (team sports)—it consistently outperforms strong single- and multi-agent baselines, with the largest gains in genuinely multi-agent scenarios.

Committee:
Katia Sycara (advisor)
Jiaoyang Li
Renos Zabounidis