Abstract
For the challenging pole balancing task we propose a system which uses raw visual input data for reinforcement learning to evolve a control strategy. Therefore we use a neural network – a deep autoencoder – to encode the camera images and thus the system states in a low dimensional feature space. The system is compared to controllers that work directly on the motor sensor data. We show that the performances of both systems are settled in the same order of magnitude.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Riedmiller, M., Lange, S., Voigtlaender, A.: Autonomous Reinforcement Learning on Raw Visual Input Data in a Real World Application. In: International Joint Conference on Neural Networks (2012)
Ormoneit, D., Sen, Ś.: Kernel-Based Reinforcement Learning. Mach. Learn. 49, 161–178 (2002)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)
Riedmiller, M.: Neural Fitted Q Iteration - First Experiences with a Data Efficient Neural Reinforcement Learning Method. In: Gama, J., Camacho, R., Brazdil, P.B., Jorge, A.M., Torgo, L. (eds.) ECML 2005. LNCS (LNAI), vol. 3720, pp. 317–328. Springer, Heidelberg (2005)
Riedmiller, M.: Neural Reinforcement Learning to Swing-Up and Balance a Real Pole. In: IEEE International Conference on Systems, Man and Cybernetics, pp. 3191–3196. IEEE Press, New York (2005)
Tolat, V.V., Widrow, B.: An Adaptive ’Broom Balancer’ with Visual Inputs. In: IEEE International Conference on Neural Networks, pp. 641–647 (1988)
Wenzel, L., Vazquez, N., Nair, D., Jamal, R.: Computer Vision Based Inverted Pendulum. In: Proceedings of the 17th IEEE Instrumentation and Measurement Technology Conference, pp. 1319–1323 (2000)
Wang, H., Chamroo, A., Vasseur, C., Koncar, V.: Hybrid Control for Vision Based Cart-Inverted Pendulum System. In: American Control Conference, pp. 3845–3850 (2008)
Hinton, G.E., Salakhutdinov, R.R.: Reducing the Dimensionality of Data with Neural Networks. Science 313, 504–507 (2006)
Ciresan, D.C., Meier, U., Gambardella, L.M., Schmidhuber, J.: Deep Big Simple Neural Nets Excel on Handwritten Digit Recognition. Neural Comput. 22, 3207–3220 (2010)
Katsikopoulos, K.V., Engelbrecht, S.E.: Markov Decision Processes with Delays and Asynchronous Cost Collection. IEEE Trans. Autom. Control 48, 568–574 (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Mattner, J., Lange, S., Riedmiller, M. (2012). Learn to Swing Up and Balance a Real Pole Based on Raw Visual Input Data. In: Huang, T., Zeng, Z., Li, C., Leung, C.S. (eds) Neural Information Processing. ICONIP 2012. Lecture Notes in Computer Science, vol 7667. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-34500-5_16
Download citation
DOI: https://doi.org/10.1007/978-3-642-34500-5_16
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-34499-2
Online ISBN: 978-3-642-34500-5
eBook Packages: Computer ScienceComputer Science (R0)