Finite-Element methods with local triangulation refinement for continuous Reinforcement Learning problems - Robotics Institute Carnegie Mellon University

Finite-Element methods with local triangulation refinement for continuous Reinforcement Learning problems

Remi Munos
Conference Paper, Proceedings of 9th European Conference on Machine Learning (ECML '97), pp. 170 - 182, April, 1997

Abstract

This paper presents a reinforcement learning algorithm for generating an adaptive control for a continuous process. Like Dynamic Programming methods, reinforcement learning find the optimal control by building a function, called the value function, that estimates the best expectation of future rewards. The algorithm proposed here uses finite-elements methods for approximating this function. It is composed of two dynamics: the learning dynamics, called Finite-Element Reinforcement Learning, which estimates the values at the vertices of a triangulation defined upon the state space, and the structural dynamics, which refines the triangulation inside regions where the value function is irregular. This mesh refinement algorithm intends to solve the problem of the combinatorial explosion of the number of values to be estimated. A formalism for reinforcement learning in the continuous case is proposed, the Hamilton-Jacobi-Bellman equation is stated, then the algorithm is presented and applied to a simple two-dimensional target problem.

BibTeX

@conference{Munos-1997-16481,
author = {Remi Munos},
title = {Finite-Element methods with local triangulation refinement for continuous Reinforcement Learning problems},
booktitle = {Proceedings of 9th European Conference on Machine Learning (ECML '97)},
year = {1997},
month = {April},
pages = {170 - 182},
}