|
|
|
|
RI | Publications | Gradient Descent Approaches to Neural-Net-Based Solutions of the Hamilton-Jacobi-Bellman Equation
|
|
Text only version of this site
Gradient Descent Approaches to Neural-Net-Based Solutions of the Hamilton-Jacobi-Bellman Equation
R. Munos, L. Baird, and A. Moore
International Joint Conference on Neural Networks, July, 1999.
Jump to: Download | Abstract | Notes | Text Reference | BibTeX Reference
| Download [Help] |
Adobe portable document format (pdf) [192 KB] Compressed postscript (ps.gz) [128 KB]
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.
| Abstract |
In this paper we investigate new approaches to dynamic-programming-based optimal control of continuous time-and-space systems. We use neural networks to approximate the solution to the Hamilton-Jacobi-Bellman (HJB) equation which is, in the deterministic case studied here, a first-order, non-linear, partial differential equation. We derive the gradient descent rule for integrating this equation inside the domain, given the conditions on the boundary. We apply this approach to the ``Car-on-the-hill'' which is a two-dimensional highly non-linear control problem. We discuss the results obtained and point out a low quality of approximation of the value function and of the derived control. We attribute this bad approximation to the fact that the HJB equation has many generalized solutions (i.e. differentiable almost everywhere) other than the value function, and our gradient descent method converges to one among these functions, thus possibly failing to find the correct value function. We illustrate this limitation on a simple one-dimensional control problem.
| Notes |
Associated lab/group: Auton Lab
Associated project: Auton Project
Number of pages: 6
| Text Reference |
R. Munos, L. Baird, and A. Moore, "Gradient Descent Approaches to Neural-Net-Based Solutions of the Hamilton-Jacobi-Bellman Equation," International Joint Conference on Neural Networks, July, 1999.
| BibTeX Reference |
@inproceedings{Munos_1999_2623,
author = "Remi Munos and Leemon Baird and Andrew Moore",
title = "Gradient Descent Approaches to Neural-Net-Based Solutions of the Hamilton-Jacobi-Bellman Equation",
booktitle = "International Joint Conference on Neural Networks",
month = "July",
year = "1999"
}