Carnegie Mellon University
Computational Rationalization: The Inverse Equilibrium Problem

Kevin Waugh, Brian D. Ziebart, and J. Andrew (Drew) Bagnell
Proceedings of the International Conference on Machine Learning, June, 2011.

  • Adobe portable document format (pdf) (289KB)
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Modeling the purposeful behavior of imperfect agents from a small number of observations is a challenging task. When restricted to the single-agent decision-theoretic setting, inverse optimal control techniques assume that observed behavior is an approximately optimal solution to an unknown decision problem. These techniques learn a utility function that explains the example behavior and can then be used to accurately predict or imitate future behavior in similar observed or unobserved situations. In this work, we consider similar tasks in competitive and cooperative multi-agent domains. Here, unlike single-agent settings, a player cannot myopically maximize its reward --- it must speculate on how the other agents may act to influence the game's outcome. Employing the game-theoretic notion of regret and the principle of maximum entropy, we introduce a technique for predicting and generalizing behavior, as well as recovering a reward function in these domains.

Game Theory, Inverse Optimal Control, Artificial Intelligence

Associated Project(s): Multidisciplinary University Research Initiative

Text Reference
Kevin Waugh, Brian D. Ziebart, and J. Andrew (Drew) Bagnell, "Computational Rationalization: The Inverse Equilibrium Problem," Proceedings of the International Conference on Machine Learning, June, 2011.

BibTeX Reference
   author = "Kevin Waugh and Brian D. Ziebart and J. Andrew (Drew) Bagnell",
   title = "Computational Rationalization: The Inverse Equilibrium Problem",
   booktitle = "Proceedings of the International Conference on Machine Learning",
   month = "June",
   year = "2011",