Abstract
In this paper, we employ a central pattern generator (CPG) driven radial basis function network (RBFN) based controller to learn optimized locomotion for a complex dung beetle-like robot using reinforcement learning approach called “Policy Improvement with Path Integrals (PI\(^2\))”. Our CPG driven RBFN controller is inspired by rhythmic dynamic movement primitives (DMPs). The controller can be also seen as an extension to a traditional CPG controller, which usually controls only the frequency of the motor patterns but not the shape. Our controller uses the CPG to control the frequency while the RBFN takes care of the shape of the motor patterns. In this paper, we only focus on the shape of the motor patterns and optimize those with respect to walking speed and energy efficiency. As a result, the robot can travel faster and consume less power than using only the CPG controller.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Tenreiro Machado, J.A., Silva, M.F.: An overview of legged robots. In: International Symposium on Mathematical Methods in Engineering, MME 2006 (2006). https://www.researchgate.net/publication/258972509_An_Overview_of_Legged_Robots
Bellicoso, C.D., et al.: Advances in real-world applications for legged robots. J. Field Rob. 35(8), 1311–1326 (2018). https://doi.org/10.1002/rob.21839
Görner, M., Chilian, A., Hirschmüller, H.: Towards an autonomous walking robot for planetary exploration. In: Proceedings of the 10th International Symposium on Artificial Intelligence, Robotics and Automation in Space (i-SAIRAS), September 2010. http://robotics.estec.esa.int/i-SAIRAS/isairas2010/PAPERS/036-2798-p.pdf
Ignasov, J., et al.: Bio-inspired design and movement generation of dung beetle-like legs. Artif. Life Rob. (2018). https://doi.org/10.1007/s10015-018-0475-5
Ijspeert, A.J.: Central pattern generators for locomotion control in animals and robots: a review. Neural Netw. 21(4), 642–653 (2008). https://doi.org/10.1016/j.neunet.2008.03.014
Theodorou, E., Buchli, J., Schaal, S.: Reinforcement learning of motor skills in high dimensions: a path integral approach. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 2397–2403 (2010). https://doi.org/10.1109/ROBOT.2010.5509336
Stulp, F., Schaal, S.: Hierarchical reinforcement learning with movement primitives. In: 11th IEEE-RAS International Conference on Humanoid Robots, pp. 231–238 (2011). https://doi.org/10.1109/Humanoids.2011.6100841
Chatterjee, S., et al.: Reinforcement learning approach to generate goal-directed locomotion of a snake-like robot with screw-drive units. In: 2014 23rd International Conference on Robotics in Alpe-Adria-Danube Region (RAAD) (2014). https://doi.org/10.1109/RAAD.2014.7002234
Stulp, F., Sigaud, O.: Robot skill learning: from reinforcement learning to evolution strategies. Paladyn J. Behav. Rob. 4(1), 49–61 (2013). https://doi.org/10.2478/pjbr-2013-0003
Pasemann, F., Hild, M., Zahedi, K.: SO(2)-networks as neural oscillators. In: Mira, J., Álvarez, J.R. (eds.) IWANN 2003. LNCS, vol. 2686, pp. 144–151. Springer, Heidelberg (2003). https://doi.org/10.1007/3-540-44868-3_19
Ijspeert, A.J., Nakanishi, J., Hoffmann, H., Pastor, P., Schaal, S.: Dynamical movement primitives: learning attractor models for motor behaviors. Neural Comput. 25(2), 328–373 (2013). https://doi.org/10.1162/NECO_a_00393
Ijspeert, A.J., Nakanishi, J., Schaal, S.: Learning attractor landscapes for learning motor primitives. In: Advances in Neural Information Processing Systems, Vancouver, BC, CA, vol. 15, pp. 1547–1554 (2003). https://papers.nips.cc/paper/2140-learning-attractor-landscapes-for-learning-motor-primitives.pdf
Manoonpong, P., Pasemann, F., Woergoetter, F.: Sensor-driven neural control for omnidirectional locomotion and versatile reactive behaviors of walking machines. Rob. Auton. Syst. 56(3), 265–288 (2008). https://doi.org/10.1016/j.robot.2007.07.004
Grinke, E., Tetzlaff, C., Wörgötter, F., Manoonpong, P.: Synaptic plasticity in a recurrent neural network for versatile and adaptive behaviors of a walking robot. Frontiers Neurorobotics 9 (2015). https://doi.org/10.3389/fnbot.2015.00011
Righetti, L., Ijspeert, A.J.: Programmable central pattern generators: an application to biped locomotion control. In: Proceedings 2006 IEEE International Conference on Robotics and Automation, ICRA 2006 (2006). https://doi.org/10.1109/ROBOT.2006.1641933
Acknowledgements
This research was supported partly by the Human Frontier Science Program under Grant agreement no. RGP0002/2017, Startup Grant-IST Flagship research of Vidyasirimedhi Institute of Science & Technology (VISTEC), the European Community H2020 Programme (Future and Emerging Technologies, FET) under grant agreement no. 732266, Plan4Act.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Pitchai, M. et al. (2019). CPG Driven RBF Network Control with Reinforcement Learning for Gait Optimization of a Dung Beetle-Like Robot. In: Tetko, I., Kůrková, V., Karpov, P., Theis, F. (eds) Artificial Neural Networks and Machine Learning – ICANN 2019: Theoretical Neural Computation. ICANN 2019. Lecture Notes in Computer Science(), vol 11727. Springer, Cham. https://doi.org/10.1007/978-3-030-30487-4_53
Download citation
DOI: https://doi.org/10.1007/978-3-030-30487-4_53
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-30486-7
Online ISBN: 978-3-030-30487-4
eBook Packages: Computer ScienceComputer Science (R0)