Abstract
Autonomous control of nonholonomic systems is one big challenge, because there is no unified control method that can handle any nonholonomic systems even if the dynamics are known. To this challenge, in this study, we propose a reinforcement learning (RL) approach which enables the controller to acquire an appropriate control policy even without knowing the detailed dynamics. In particular, we focus on the control problem of a free-falling cat system whose dynamics are highly-nonlinear and nonholonomic. To accelerate the learning, we take the policy gradient method that exploits the basic knowledge of the system, and present an appropriate policy representation for the task. It is shown that this RL method achieves remarkably faster learning than that by the existing genetic algorithm-based method.
Keywords
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Nakamura, Y.: Nonholonomic robot systems, Part 1: what’s a nonholonomic robot? Journal of RSJ 11, 521–528 (1993)
Brockett, R.W.: Asymptotic stability and feedback stabilization. Progress in Mathematics 27, 181–208 (1983)
Mita, T.: Introduction to nonlinear control Theory-Skill control of underactuated robots. SHOKODO Co., Ltd. (2000) (in Japanese)
Murray, R.M., Sastry, S.S.: Nonholonomic motion planning: steering using sinusoids. IEEE Transactions on Automatic Control 38, 700–716 (1993)
Holamoto, S., Funasako, T.: Feedback control of a planar space robot using a moving manifold. Journal of RSJ 25, 745–751 (1993)
Peters, J., Schaal, S.: Reinforcement learning of motor skills with policy gradients. Neural Networks 21, 682–697 (2008)
Miyamae, A., et al.: Instance-based policy learning by real-coded genetic algorithms and its application to control of nonholonomic systems. Transactions of the Japanese Society for Artificial Intelligence 24, 104–115 (2009)
Tsuchiya, C., et al.: SLIP: A sophisticated learner for instance-based policy using hybrid GA. Transactions of SICE 42, 1344–1352 (2006)
Nakamura, Y., Mukherjee, R.: Nonholonomic path planning of space robots via a bidirectional approach. IEEE Transactions on Robotics and Automation 7, 500–514 (1991)
Baxter, J., Bartlett, P.L.: Infinite-horizon policy-gradient estimation. Journal of Artificial Intelligence Research 15, 319–350 (2001)
Ge, X., Chen, L.: Optimal control of nonholonomic motion planning for a free-falling cat. Applied Mathematics and Mechanics 28, 601–607 (2007)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Nakano, D., Maeda, Si., Ishii, S. (2012). Control of a Free-Falling Cat by Policy-Based Reinforcement Learning. In: Villa, A.E.P., Duch, W., Érdi, P., Masulli, F., Palm, G. (eds) Artificial Neural Networks and Machine Learning – ICANN 2012. ICANN 2012. Lecture Notes in Computer Science, vol 7553. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-33266-1_15
Download citation
DOI: https://doi.org/10.1007/978-3-642-33266-1_15
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-33265-4
Online ISBN: 978-3-642-33266-1
eBook Packages: Computer ScienceComputer Science (R0)