Learning to Select State Machines using Expert Advice on an Autonomous Robot

Brenna Argall, Brett Browning, and Manuela Veloso
Proceedings IEEE International Conference on Robotics and Automation, May, 2007.


Download
  • Adobe portable document format (pdf) (232KB)
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Abstract
Hierarchical state machines have proven to be a powerful tool for controlling autonomous robots due to their flexibility and modularity. For most real robot implementations, however, it is often the case that the control hierarchy is hand-coded. As a result, the development process is often time intensive and error prone. In this paper, we explore the use of an experts learning approach, based on Auer and colleagues' Exp3, to help overcome some of these limitations. In particular, we develop a modified learning algorithm, which we call rExp3, that exploits the structure provided by a control hierarchy by treating each state machine as an 'expert'. Our experiments validate the performance of rExp3 on a real robot performing a task, and demonstrate that rExp3 is able to quickly learn to select the best state machine expert to execute. Through our investigations in these environments, we identify a need for faster learning recovery when the relative performances of experts reorder, such as in response to a discrete environment change. We introduce a modified learning rule to improve the recovery rate in these situations and demonstrate through simulation experiments that rExp3 performs as well or better than Exp3 under such conditions.

Notes

Text Reference
Brenna Argall, Brett Browning, and Manuela Veloso, "Learning to Select State Machines using Expert Advice on an Autonomous Robot," Proceedings IEEE International Conference on Robotics and Automation, May, 2007.

BibTeX Reference
@inproceedings{Argall_2007_6232,
   author = "Brenna Argall and Brett Browning and Manuela Veloso",
   title = "Learning to Select State Machines using Expert Advice on an Autonomous Robot",
   booktitle = "Proceedings IEEE International Conference on Robotics and Automation",
   month = "May",
   year = "2007",
}