Finite-Element methods with local triangulation refinement for continuous Reinforcement Learning problems

Remi Munos
European Conference on Machine Learning 1997, 1997.


Download
  • Adobe portable document format (pdf) (569KB)
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Abstract
This paper presents a reinforcement learning algorithm for generating an adaptive control for a continuous process. Like Dynamic Programming methods, reinforcement learning find the optimal control by building a function, called the value function, that estimates the best expectation of future rewards. The algorithm proposed here uses finite-elements methods for approximating this function. It is composed of two dynamics: the learning dynamics, called Finite-Element Reinforcement Learning, which estimates the values at the vertices of a triangulation defined upon the state space, and the structural dynamics, which refines the triangulation inside regions where the value function is irregular. This mesh refinement algorithm intends to solve the problem of the combinatorial explosion of the number of values to be estimated. A formalism for reinforcement learning in the continuous case is proposed, the Hamilton-Jacobi-Bellman equation is stated, then the algorithm is presented and applied to a simple two-dimensional target problem.

Notes
Associated Lab(s) / Group(s): Auton Lab
Associated Project(s): Auton Project
Number of pages: 12

Text Reference
Remi Munos, "Finite-Element methods with local triangulation refinement for continuous Reinforcement Learning problems," European Conference on Machine Learning 1997, 1997.

BibTeX Reference
@inproceedings{Munos_1997_2944,
   author = "Remi Munos",
   title = "Finite-Element methods with local triangulation refinement for continuous Reinforcement Learning problems",
   booktitle = "European Conference on Machine Learning 1997",
   year = "1997",
}