Variable Resolution Dynamic Programming: Efficiently Learning Action Maps in Multivariate Real-valued State-spaces

Abstract

An effective method to create an autonomous reactive controller is to learn a model of the environment and then use dynamic programming to derive a policy to maximize long term reward. Neither learning environmental models nor dynamic programming require parametric assumptions about the world, and so learning can proceed with no danger of becoming “stuck― by a mismatch between the parametric assumptions and reality. The paper discusses how such an approach can be realized in real valued multivariate state spaces in which straightforward discretization falls prey to the curse of dimensionality.