Planning for distributed execution through use of probabilistic opponent models

Abstract

In multiagent domains with adversarial and cooperative team agents, team agents should be adaptive to the current environment and opponent. We introduce an online method to provide the agents with team plans that a "coach" agent generates in response to the specific opponents. The coach agent can observe the agents’ behaviors but it has only periodic communication with the rest of the team. The coach uses a Simple Temporal Network to represent team plans as coordinated movements among the multiple agents and the coach searches for an opponent-dependent plan for its teammates. This plan is then communicated to the agents, who execute the plan in a distributed fashion, using information from the plan to maintain consistency among the team members. In order for these plans to be effective and adaptive, models of opponent movement are used in the planning. The coach is then able to quickly select between different models online by using a Bayesian style update on a probability distribution over the models. Planning then uses the model which is found to be the most likely. The system is fully implemented in a simulated robotic soccer environment. In several recent games with completely unknown adversarial teams, the approach demonstrated a visible adaptation to the different teams.