Abstract
This paper considers reinforcement learning control with the self-organizing map. Reinforcement learning uses the observable states of objective system and signals from interaction of the system and environment as input data. For fast learning in neural network training, it is necessary to reduce learning data. In this paper, we use the self-organizing map to partition the observable states. Partitioning states reduces the number of learning data which is used for training neural networks. And neural dynamic programming design method is used for the controller. For evaluating the designed reinforcement learning controller, a double linked inverted pendulum on the cart system is simulated. The designed controller is composed of serial connection of self-organizing map and two Multi-layer Feed-Forward Neural Networks.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)
Anderson, C.W.: Strategy Learning with Multilayer Connectionist Representations. In: Proceedings of the 4th International Workshop on Machine Learning, pp. 103–114 (1987)
Anderson, C.W.: Learning to Control an Inverted Pendulum Using Neural Network. IEEE Control Systems Magazine 9(3), 31–37 (1989)
Barto, A.G., Sutton, R.S., Anderson, C.W.: Neuronlike Adaptive Elements That Can Solve Difficult Learning Control Problems. IEEE Transactions on Systems, Man, and Cybernetics 13(5) (1983)
Albus, J.S.: A New Approach to Manipulator control: The Cerebellar Model Articulation Controller (CMAC). Journal of Dynamics Systems, Measurement, and Control, 220–227 (1975)
Hougen, D.F., Gini, M., Slagle, J.: Partitioning input space for reinforcement learning for control. In: Proceedings of the IEEE International Conference on Robotics and Automation, April 1996, pp. 1917–1922 (1996)
Smith, A.J.: Applicatoins of the self-organising map to reinforcement learning. Neural Networks (Special Issue) 15, 1107–1124 (2002)
Kohonen, T.: Self organising maps. Springer, Heidelberg (2001)
Sutton, R.S.: Learning to predict by the methods of temporal difference. Machine Learning 3, 9–44 (1988)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer Berlin Heidelberg
About this paper
Cite this paper
Lee, Jk., Oh, Ts., Shin, Ys., Yoon, Tj., Kim, Ih. (2007). Postural Control of Two-Stage Inverted Pendulum Using Reinforcement Learning and Self-organizing Map. In: Beliczynski, B., Dzielinski, A., Iwanowski, M., Ribeiro, B. (eds) Adaptive and Natural Computing Algorithms. ICANNGA 2007. Lecture Notes in Computer Science, vol 4432. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-71629-7_81
Download citation
DOI: https://doi.org/10.1007/978-3-540-71629-7_81
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-71590-0
Online ISBN: 978-3-540-71629-7
eBook Packages: Computer ScienceComputer Science (R0)