The Robotics Institute

Carnegie Mellon Robotics Institute

Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Abstract

This paper presents a reinforcement learning algorithm for generating an adaptive control for a continuous process. Like Dynamic Programming methods, reinforcement learning find the optimal control by building a function, called the value function, that estimates the best expectation of future rewards. The algorithm proposed here uses finite-elements methods for approximating this function. It is composed of two dynamics: the learning dynamics, called Finite-Element Reinforcement Learning, which estimates the values at the vertices of a triangulation defined upon the state space, and the structural dynamics, which refines the triangulation inside regions where the value function is irregular. This mesh refinement algorithm intends to solve the problem of the combinatorial explosion of the number of values to be estimated. A formalism for reinforcement learning in the continuous case is proposed, the Hamilton-Jacobi-Bellman equation is stated, then the algorithm is presented and applied to a simple two-dimensional target problem.

Notes

Associated Lab(s) / Group(s): Auton Lab

Associated Project(s): Auton Project

Number of pages: 12

Text Reference

Remi Munos, "Finite-Element methods with local triangulation refinement for continuous Reinforcement Learning problems," European Conference on Machine Learning 1997, 1997.

BibTeX Reference

@inproceedings{Munos_1997_2944,
   author = "Remi Munos",
   title = "Finite-Element methods with local triangulation refinement for continuous Reinforcement Learning problems",
   booktitle = "European Conference on Machine Learning 1997",
   year = "1997",
}

The Robotics Institute

Menu