Abstract
Hexapod robots are highly flexible and statically stable platforms that are able to traverse a wide range of terrains. The overactuated nature of the platform results in a difficult and high dimensional locomotion problem. Despite the dimensionality, the locomotion task exhibits both morphological and temporal structure. Procedural locomotion generation uses inverse kinematics and leg scheduling to move the torso towards a target location and is used extensively in graphical animation. However, kinematic approaches often fail in dynamical environments due to slippages and other unforseen effects. We propose taking such an approach and replacing the leg scheduling and other heuristic decision points with neural networks. We show that we can formulate the gait-phase decision for each leg as learnable state machines and use Evolutionary Strategies (ES) in simulation to optimize the free parameters. We also show that other useful locomotion traits, such as torso and leg height can be optimized as well. The result improves on the baseline procedural approach on difficult terrains such as stairs. Our approach attempts to inject learnable experience from simulation into an algorithm with a high inductive prior, resulting in a hybrid control policy that has superior performance on a wide range of uneven terrains.
The research leading to these results has received funding from the Czech Science Foundation under Project 20-29531S. This work has been supported by the EU OP RDE funded project Research Center for Informatics; reg. No.: CZ.02.1.01/0.0./0.0./16_019/0000765. T.Azayev was also partially supported by Grant Agency of the CTU in Prague under Project SGS22/111/OHK3/2T/13.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Video: https://vimeo.com/744114592.
References
Azayev, T., Zimmerman, K.: Blind hexapod locomotion in complex terrain with gait adaptation using deep reinforcement learning and classification. J. Intell. Robot. Syst. 99(3), 659–671 (2020). https://doi.org/10.1007/s10846-020-01162-8
Belter, D., Skrzypczynski, P.: A biologically inspired approach to feasible gait learning for a hexapod robot. Appl. Math. Comput. Sci. 20, 69–84 (2010). https://doi.org/10.2478/v10006-010-0005-7
Bjelonic, M., Kottege, N., Beckerle, P.: Proprioceptive control of an over-actuated hexapod robot in unstructured terrain. In: 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2042–2049. IEEE (2016). https://doi.org/10.1109/IROS.2016.7759321
Campos, R., Matos, V., Santos, C.: Hexapod locomotion. In: IECON 2010–36th Annual Conference on IEEE Industrial Electronics Society, pp. 1546–1551. IEEE, Glendale, AZ, USA (2010)
Cizek, P., Masri, D., Faigl, J.: Foothold placement planning with a hexapod crawling robot. In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4096–4101. IEEE, Vancouver, BC, Canada (2017)
Karim, A.A., Gaudin, T., Meyer, A., Buendia, A., Bouakaz, S.: Procedural locomotion of multilegged characters in dynamic environments. Comput. Anim. Virtual Worlds 24(1), 3–15 (2013). https://doi.org/10.1002/cav.1467
Lele, A.S., Fang, Y., Ting, J., Raychowdhury, A.: Online reward-based training of spiking central pattern generator for hexapod locomotion. In: 2020 IFIP/IEEE 28th International Conference on Very Large Scale Integration (VLSI-SOC), pp. 208–209 (2020). https://doi.org/10.1109/VLSI-SOC46417.2020.9344100
Loshchilov, I., Hutter, F.: CMA-ES for hyperparameter optimization of deep neural networks. CoRR abs/1604.07269 (2016)
Mania, H., Guy, A., Recht, B.: Simple random search of static linear policies is competitive for reinforcement learning. In: Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., Garnett, R. (eds.) Advances in Neural Information Processing Systems. vol. 31. Curran Associates, Inc. (2018). https://doi.org/10.48550/arXiv. 1803.07055
Minati, L., Frasca, M., Yoshimura, N., Koike, Y.: Versatile locomotion control of a hexapod robot using a hierarchical network of nonlinear oscillator circuits. IEEE Access 6, 8042–8065 (2018). https://doi.org/10.1109/ACCESS.2018.2799145
Schilling, M., Konen, K., Ohl, F.W., Korthals, T.: Decentralized deep reinforcement learning for a distributed and adaptive locomotion controller of a hexapod robot. In: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 5335–5342. IEEE (2020). https://doi.org/10.1109/IROS45743.2020.9341754
Todd, D.J.: Walking Machines: An Introduction to Legged Robots. Springer, Cham (2013)
Vice, J., Sukthankar, G., Douglas, P.K.: Leveraging evolutionary algorithms for feasible hexapod locomotion across uneven terrain. arXiv preprint arXiv:2203.15948 (2022). 10.48550/arXiv. 2203.15948
Wang, T., Liao, R., Ba, J., Fidler, S.: Nervenet: learning structured policy with graph neural networks. In: International Conference on Learning Representations (2018)
Zangrandi, M., Arrigoni, S., Braghin, F.: Control of a hexapod robot considering terrain interaction. CoRR abs/2112.10206 (2021). https://doi.org/10.48550/arXiv.2112.10206
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Azayev, T., Hronovský, J., Zimmermann, K. (2023). Improving Procedural Hexapod Locomotion Generation with Neural Network Decision Modules. In: Mazal, J., et al. Modelling and Simulation for Autonomous Systems. MESAS 2022. Lecture Notes in Computer Science, vol 13866. Springer, Cham. https://doi.org/10.1007/978-3-031-31268-7_8
Download citation
DOI: https://doi.org/10.1007/978-3-031-31268-7_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-31267-0
Online ISBN: 978-3-031-31268-7
eBook Packages: Computer ScienceComputer Science (R0)