Abstract
This paper presents a combined framework with nonlinear model predictive control (NMPC) reinforcement learning (RL) for locomotion of a legged robot. A neural network trained by RL works as a footstep planner which decides where to put the feet of the robot on the ground. Given the constraints of footsteps and dynamics of the model, ground reaction forces exerting on each legs are obtained through NMPC and applied to the robot. This framework increases sample efficiency compared to the end-to-end RL and shows better performances than base NMPC controller which decides its footsteps in a heuristic manner. The proposed framework is verified on a simulation environment by performing challenging tasks such as push recovery and rough terrain walking.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Kuindersma, S., et al.: Optimization-based locomotion planning, estimation, and control design for the atlas humanoid robot. Auton. Robots 40(3), 429–455 (2015). https://doi.org/10.1007/s10514-015-9479-3
Hong, S., Kim, J.-H., Park, H.-W.: Real-time constrained nonlinear model predictive control on SO(3) for dynamic legged locomotion. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 3982–3989 (2020)
Ding, Y., Pandala, A., Park, H.-W., et al.: Representation-free model predictive control for dynamic motions in quadrupeds. IEEE Trans. Robot. 37, 1154–1171 (2021)
Carlo, J., Wensing, P.M., Kim, S., et al.: Dynamic locomotion in the MIT cheetah 3 through convex model-predictive control. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1–9 (2018)
Raibert, M.H., Brown, H.B., Chepponis, M.: Experiments in balance with a 3D one-legged hopping machine. Int. J. Robot. Res. 3, 75–92 (1984)
Pratt, J., Carff, J., Goswami, A., et al.: Capture point: a step toward humanoid push recovery. In: IEEE-RAS International Conference on Humanoid Robots, vol. 6, pp. 200–207 (2006)
Mordatch, I., Todorov, E., Popovic, Z.: Discovery of complex behaviors through contact-invariant optimization. ACM Trans. Graph. 31(4), 1–8 (2012)
Winkler, A.W., Bellicoso, C.D., Hutterm, M., et al.: Gait and trajectory optimization for legged systems through phase-based end-effector parameterization. IEEE Robot. Autom. Lett. 3, 1560–1567 (2018)
Li, C., Ding, Y., Park, H.-W.: Centroidal-momentum-based trajectory generation for legged locomotion. Mechatronics 68, 102364 (2020)
Hwangbo, J., Lee, J., Hutter, M., et al.: Learning agile and dynamic motor skills for legged robots. Sci. Robot. 4, 26 (2019)
Carius, J., Farshidian, F., Hutter, M.: MPC-Net: a first principles guided policy search. IEEE Robot. Autom. Lett. 5, 2897–2904 (2020)
Peng, X.B., Coumans, E., Levine, S., et al.: Learning agile robotic locomotion skills by imitating animals. arXiv:2004.00784 (2020)
Mahony, R., Hamel, T., Pflimlin, J.: Nonlinear complementary filters on the special orthogonal group. IEEE Trans. Autom. Control 53(5), 1203–1218 (2008)
Kim, D., Carlo, J., Kim, S., et al.: Highly dynamic quadruped locomotion via whole-body impulse control and model predictive control. arXiv:1909.06586 (2019)
Schulman, J., Wolski, F., Klimov, O., et al.: Proximal policy optimization algorithm. arXiv:1707. 06347 (2017)
Tan, J., Coumans, E., Vanhoucke, V., et al.: Sim-to-real: learning agile locomotino for quadruped robots. arXiv:1804.10332 (2018)
Hwangbo, J., Lee, J., Hutter, M.: Per-contact iteration method for solving contact dynamics. IEEE Robot. Autom. Lett. 3(2), 895–902 (2018)
Tsounis, V., Alge, M., Hutter, M. et al.: DeepGait: planning and control of quadrupedal gaits using deep reinforcement learning. arXiv:1909.08339 (2019)
Yang, Y., Zhang, T, Boots, B., et al.: Fast and efficient locomotion via learned gait transitions. arXiv:2104.04644 (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Byun, J.W., Youm, D., Jeon, S., Hwangbo, J., Park, HW. (2022). Learning Footstep Planning for the Quadrupedal Locomotion with Model Predictive Control. In: Kim, J., et al. Robot Intelligence Technology and Applications 6. RiTA 2021. Lecture Notes in Networks and Systems, vol 429. Springer, Cham. https://doi.org/10.1007/978-3-030-97672-9_4
Download citation
DOI: https://doi.org/10.1007/978-3-030-97672-9_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-97671-2
Online ISBN: 978-3-030-97672-9
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)