Abstract
The locomotion of legged robot is often controlled by predefined gaits, and this approach works well when all joints and motors are operating normally. However, walking legged robots usually have high risk of being damaged during operation, causing the breakdown of the robotic joints. In this paper, we introduce a reinforcement learning based approach for the legged robot to generate real-time locomotion response to the emergence of locomotion breakdown. Our approach detects the functionality of the available joints, substitutes the pre-defined gaits with proper gait function accordingly, and upgrades the gait-generation function by Q-Learning for the proper locomotion.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Eason, G., Noble, B., Sneddon, I.N.: On certain integrals of Lipschitz-Hankel type involving products of Bessel functions. Philos. Trans. R. Soc. Lond. Ser. A Math. Phys. Sci. 247(935), 529–551 (1955)
Sutton, R.S., Barto, A.G.: Introduction to Reinforcement Learning, vol. 135. MIT Press, Cambridge (1998)
Watkins, C.J., Dayan, P.: Q-learning. Mach. Learn. 8(3–4), 279–292 (1992)
Borkar, V.S., Meyn, S.P.: The ODE method for convergence of stochastic approximation and reinforcement learning. SIAM J. Control Optim. 38(2), 447–469 (2000)
Auslander, B., Lee-Urban, S., Hogg, C., Muñoz-Avila, H.: Recognizing the enemy: combining reinforcement learning with strategy selection using case-based reasoning. In: Althoff, K.-D., Bergmann, R., Minor, M., Hanft, A. (eds.) ECCBR 2008. LNCS (LNAI), vol. 5239, pp. 59–73. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-85502-6_4
Bianchi, R.A., Ribeiro, C.H., Costa, A.H.: Accelerating autonomous learning by using heuristic selection of actions. J. Heuristics 14(2), 135–168 (2008)
Bianchi, R.A.C., Ros, R., Lopez de Mantaras, R.: Improving reinforcement learning by using case based heuristics. In: McGinty, L., Wilson, David C. (eds.) ICCBR 2009. LNCS (LNAI), vol. 5650, pp. 75–89. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-02998-1_7
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT press, Cambridge (2018)
Bianchi, R.A.C.: Using heuristics to accelerate reinforcement learning algorithms. Dissertation Ph.D. thesis, University of São Paulo (2004)
Yu, T.K., Yang, M.C., Samani, H.: Reinforcement learning and convolutional neural network system for firefighting rescue robot. In: MATEC Web of Conferences, vol. 161, p. 03028 (2018)
Samani, H., Zhu, R.: Robotic automated external defibrillator ambulance for emergency medical service in smart cities. IEEE Access 4, 268–283 (2016)
Samani, H.: Cognitive Robotics. CRC Press, Boca Raton (2015)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Yang, MC., Samani, H., Zhu, K. (2019). Emergency-Response Locomotion of Hexapod Robot with Heuristic Reinforcement Learning Using Q-Learning. In: Ronzhin, A., Rigoll, G., Meshcheryakov, R. (eds) Interactive Collaborative Robotics. ICR 2019. Lecture Notes in Computer Science(), vol 11659. Springer, Cham. https://doi.org/10.1007/978-3-030-26118-4_31
Download citation
DOI: https://doi.org/10.1007/978-3-030-26118-4_31
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-26117-7
Online ISBN: 978-3-030-26118-4
eBook Packages: Computer ScienceComputer Science (R0)