Neural Networks with Online Sequential Learning Ability for a Reinforcement Learning Algorithm
Reinforcement learning (RL) algorithms that employ neural networks as function approximators have proven to be powerful tools for solving optimal control problems. However, neural network function approximators suffer from a number of problems like learning becomes difficult when the training data are given sequentially, difficult to determine structural parameters, and usually result in local minima or overfitting. In this paper, a novel on-line sequential learning evolving neural network model design for RL is proposed. We explore the use of minimal resource allocation neural network (mRAN), and develop a mRAN function approximation approach to RL systems. Potential of this approach is demonstrated through a case study. The mean square error accuracy, computational cost, and robustness properties of this scheme are compared with static structure neural networks.
KeywordsReinforcement learning online sequential learning resource allocation neural network
Unable to display preview. Download preview PDF.
- 1.Sutton, R.S., Barto, A.G., Williams, R.J.: Reinforcement learning is direct adaptive optimal control. IEEE Control Syst. Mag. 12(2), 19–22Google Scholar
- 2.Sutton, R.S., Barto, A.G.: Reinforcement learning: An introduction. MIT Press, CambridgeGoogle Scholar
- 3.Watkins CJCHLearning with delayed rewards. Ph. D. Thesis, University of Cambridge (1989)Google Scholar
- 16.Shiraga, N., Ozawa, S., Abe, S.: A reinforcement learning algorithm for neural networks with incremental learning ability. In: Proceeding of the 9th International Conference on Neural Information Processing, vol. 5, pp. 2566–2570 (2002)Google Scholar
- 17.Kobayashi, M., Zamani, A., Ozawa, S., Abe, S.: Reducing computations in incremental learning for feed-forward neural network with long-term memory. In: Proc. International Joint Conference on Neural Networks, pp. 1989–1994 (2001)Google Scholar