Stabilizing Human Control Strategies through Reinforcement Learning

Michael Nechyba and J. Andrew (Drew) Bagnell
April, 1999.

In attempting to build advanced robots with sophisticated in- telligent behaviors, the modern roboticist does not have to look far to find examples of such behavior. Humans are, and for the foreseeable future remain our best and only example of true intel- ligence. In comparison, even advanced robots are still embar- rassingly stupid. Consequently, one popular approach for imparting intelligent behaviors to robots and other machines ab- stracts models of human control strategy (HCS), learned directly from human control data. This type of approach can be broadly classified as "learning through observation." A competing ap- proach, which builds up complex behaviors through exploration and optimization over time, is reinforcement learning. We seek to unite these two approaches, previously considered disparate, and show that each approach, in fact, complements the other. Specif- ically, we propose a new algorithm, rooted in reinforcement learning, for stabilizing learned models of human control strate- gy. In this paper, we first describe the real-time driving simulator which we have developed for investigating human control strate- gies. Next, we motivate and describe our framework for modeling human control strategies. We then illustrate how the resulting HCS models can be stabilized through reinforcement learning and finally report some positive experimental results with the pro- posed algorithm.


Text Reference
Michael Nechyba and J. Andrew (Drew) Bagnell, "Stabilizing Human Control Strategies through Reinforcement Learning," April, 1999.

BibTeX Reference
   author = "Michael Nechyba and J. Andrew (Drew) Bagnell",
   title = "Stabilizing Human Control Strategies through Reinforcement Learning",
   booktitle = "",
   month = "April",
   year = "1999",
   number= "CMU-RI-TR-",