Reinforcement Planning: RL for Optimal Planners

Matthew Zucker and J. Andrew (Drew) Bagnell
IEEE International Conference on Robotics and Automation (ICRA), March, 2012.


Download
  • Adobe portable document format (pdf) (2MB)
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Abstract
Search based planners such as A* and Dijkstra’s algorithm are proven methods for guiding today’s robotic systems. Although such planners are typically based upon a coarse approximation of reality, they are nonetheless valuable due to their ability to reason about the future, and to generalize to previously unseen scenarios. However, encoding the desired behavior of a system into the underlying cost function used by the planner can be a tedious and error-prone task. We introduce Reinforcement Planning, which extends gradient based reinforcement learning algorithms to automatically learn useful surrogate cost functions for optimal planners. Reinforcement Planning presents several advantages over other learning approaches to planning in that it is not limited by the expertise of a human demonstrator, and that it acknowledges the domain of the planner is a simplified model of the world. We demonstrate the effectiveness of our method in learning to solve a noisy physical simulation of the well-known “marble maze” toy.

Notes
Number of pages: 6

Text Reference
Matthew Zucker and J. Andrew (Drew) Bagnell, "Reinforcement Planning: RL for Optimal Planners ," IEEE International Conference on Robotics and Automation (ICRA), March, 2012.

BibTeX Reference
@inproceedings{Zucker_2012_6997,
   author = "Matthew Zucker and J. Andrew (Drew) Bagnell",
   title = "Reinforcement Planning: RL for Optimal Planners ",
   booktitle = "IEEE International Conference on Robotics and Automation (ICRA)",
   month = "March",
   year = "2012",
   number= "CMU-RI-TR-",
}