Graphics enhanced version of this site

Emergent Hierarchical Control Structures: Learning Reactive / Hierarchical Relationships in Reinforcement Environments
B. Digney
Proceedings of the Fourth Conference on the Simulation of Adaptive Behavior: SAB 98, 1996.

Jump to: Download | Abstract | Text Reference | BibTeX Reference

Adobe portable document format (pdf) [312 KB]
Compressed postscript (ps.gz) [129 KB]

Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

The use of externally imposed hierarchical structures to reduce the complexity of learning control is common. However, it is acknowledged that learning the hierarchical structure itself is an important step towards more general (learning of many things as required) and less bounded (learning of a single thing as specified) learning. Presented in this paper is a reinforcement learning algorithm called Nested Q-learning that generates a hierarchical control structure in reinforcement learning domains. The emergent structure combined with learned bottom-up reactive reactions results in a reactive hierarchical control system. Effectively, the learned hierarchy decomposes what would otherwise be a monolithic evaluation function into many smaller evaluation functions that can be recombined without the loss of previously learned information.

B. Digney, "Emergent Hierarchical Control Structures: Learning Reactive / Hierarchical Relationships in Reinforcement Environments," Proceedings of the Fourth Conference on the Simulation of Adaptive Behavior: SAB 98, 1996.

@inproceedings{Digney_1996_3151,
   author = "Bruce Digney",
   title = "Emergent Hierarchical Control Structures: Learning Reactive / Hierarchical Relationships in Reinforcement Environments",
   booktitle = "Proceedings of the Fourth Conference on the Simulation of Adaptive Behavior: SAB 98",
   year = "1996"
}