A Modular Hierarchical Reinforcement Learning Algorithm
How to improve the learning efficiency and optimize the encapsulation of subtasks is a key problem that hierarchical reinforcement learning needs to solve. This paper proposes a modular hierarchical reinforcement learning al-gorithm, named MHRL, in which the modularized hierarchical subtasks are trained by their independent reward systems. During learning, the MHRL pro-duces an optimization strategy for different modular layers, which makes inde-pendent modules be able to concurrently execute. In addition, this paper pre-sents some experimental results for solving application problems with nested learning processes. The results show that the MHRL can increase learning reus-ability and improve learning efficiency dramatically.
KeywordsModular Hierarchical Reinforcement Learning MAXQ Markov Decision Software Reuse Optimization Algorithm
Unable to display preview. Download preview PDF.
- 1.Sutton, R.S., Barto, A.G.: Introduction to Reinforcement Learning. MIT Press, Cambridge (1998)Google Scholar
- 3.Parr, R.: Hierarchical Control and Learning for Markov Decision Processes. University of California, Berkeley (1998)Google Scholar
- 4.Dayan, P., Hinton: Feudal Reinforcement Learning. Advances in Neural Information Processing Systems 5, 271–278 (1993)Google Scholar
- 5.Hengst, B.: Discovering Hierarchical Reinforcement learning, Sydney, University of New South Wales, Australia (2003)Google Scholar
- 9.Watins, P., Dayan, N.: Q-learning. Machine Learning 8, 279–292 (1992)Google Scholar