Abstract
The article presents a new approach to the problem of a discrete neural control of an underactuated system, using reinforcement learning method to an on-line adaptation of a neural network. The controlled system is of the ball and beam type, which is the nonlinear dynamical object with the number of control signals smaller than the number of degrees of freedom. The main part of the neural control system is the actor-critic structure, that comes under the Neural Dynamic Programming algorithms family, realised in the form of Dual Heuristic Dynamic Programming structure. The control system includes moreover the PD controller and the supervisory therm, derived from the Lyapunov stability theorem, that ensures stability. The proposed neural control system works on-line and does not require a preliminary learning. Computer simulations have been conducted to illustrate the performance of the control system.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Barto, A.G., Sutton, R.S., Anderson, C.W.: Neuronlike Adaptive Elements that Can Solve Difficult Learning Control Problems. IEEE Transactions on Systems, Man and Cybernetics 13, 834–846 (1983)
Blajer, W., Kolodziejczyk, K.: Contol of Underactuated Mechanical Systems with Servo-Constraints. Nonlinear Dynamics 50, 781–791 (2007)
Eaton, P.H., Prokhorov, D.V., Wunsch, D.C.: Neurocontroller Alternatives for Fuzzy Ball-and-Beam Systems With Nonuniform Nonlinear Friction. IEEE Transactions on Neural Networks and Learning Systems 11, 423–435 (2000)
Burghardt, A., Giergiel, J.: Modelling and Control of an Underactuated Sphere and Beam System. Communications in Nonlinear Science and Numerical Simulation 16, 2350–2354 (2011)
Hendzel, Z., Burghardt, A.: Adaptive Neural Network Control of Underactuated System. In: 4th International Conference on Neural Computation Theory and Applications, pp. 505–509. SciTePress, Barcelona (2012)
Hendzel, Z., Szuster, M.: Discrete Model-Based Dual Heuristic Programming in Wheeled Mobile Robot Control. In: 10th International Conference on Dynamical Systems - Theory and Applications, Lodz, pp. 745–752 (2009)
Hendzel, Z., Szuster, M.: Discrete Model-Based Adaptive Critic Designs in Wheeled Mobile Robot Control. In: Rutkowski, L., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2010, Part II. LNCS (LNAI), vol. 6114, pp. 264–271. Springer, Heidelberg (2010)
Keshmiri, M., Jahromi, A.F., Mohebbi, A., Amoozgar, M.H., Xie, W.F.: Modeling and Control of Ball and Beam System Using Model Based and Non-model Based Control Approaches. International Journal on Smart Sensing and Intelligent Systems 5, 14–35 (2012)
Powell, W.B.: Approximate Dynamic Programming: Solving the Curses of Dimensionality. Wiley Interscience, Princeton (2007)
Prokhorov, D., Wunch, D.: Adaptive Critic Designs. IEEE Transactions on Neural Networks 8, 997–1007 (1997)
Si, J., Barto, A.G., Powell, W.B., Wunsch, D.: Handbook of Learning and Approximate Dynamic Programming. IEEE Press, Wiley-Interscience (2004)
Spong, M.W.: Modeling and control of elastic joint robot. Journal of Dynamic Systems, Measurement, and Control 109, 310–319 (1987)
Spong, M.W.: Underactuated Mechanical Systems. In: Siciliano, B., Valavanis, K.P. (eds.) Control Problems in Robotics and Automation. LNCIS, vol. 230, pp. 135–150. Springer, Heidelberg (1998)
Wen, Y.: Nonlinear PD Regulation for Ball and Beam System. International Journal of Electrical Engineering Education 46, 59–73 (2009)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Hendzel, Z., Burghardt, A., Szuster, M. (2013). Reinforcement Learning in Discrete Neural Control of the Underactuated System. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds) Artificial Intelligence and Soft Computing. ICAISC 2013. Lecture Notes in Computer Science(), vol 7894. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-38658-9_6
Download citation
DOI: https://doi.org/10.1007/978-3-642-38658-9_6
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-38657-2
Online ISBN: 978-3-642-38658-9
eBook Packages: Computer ScienceComputer Science (R0)