Abstract
The paper considers the solution to two control optimization problems for a capsule-type robot with an opposing spring. The first problem focuses on optimizing the average velocity of the robot while the other one deals with a time-optimal control problem. The capsule-type robot is an essentially nonlinear system that performs cyclic reciprocating motions to move in any direction. In this study, the usage of Reinforcement Learning and Nonlinear Model Predict approaches for such a nonlinear system is researched. The trajectories and control laws are obtained, and the results are compared with the optimization results for a simple periodic piece-wise constant law with one switch. The study was conducted by methods of mathematical modeling.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Chernousko, F.L.: Analysis and optimization of the motion of a body controlled by a movable internal mass. J. Appl. Math. Mech. 70(6), 915–941 (2006)
Zimmermann, K., Zeidis, I., Bolotnik, N., Pivovarov, M.: Dynamics of a two-module vibration-driven system moving along a rough horizontal plane. Multibody Syst. Dyn. 22, 199–219 (2009)
Fang, H.B., Xu, J.: Dynamic analysis and optimization of a three-phase control mode of a mobile system with an internal mass. J. Vib. Control 74(4), 443–451 (2011)
Yan, Y., Liu, Y., Liao, M.: A comparative study of the vibro-impact capsule systems with one-sided and two-sided constraints. Nonlinear Dyn. 89, 1063–1087 (2015)
Nunuparov, A., Becker, F., Bolotnik, N., Zeidis, I., Zimmermann, K.: Dynamics and motion control of a capsule robot with an opposing spring. Arch. Appl. Mech. 1–16 (2019)
Moritz Diehl, Rolf Findeisen, S Schwarzkopf, I Uslu, Frank Allgöwer, Hans Bock, E Gilles, and J Schlöder. An efficient algorithm for nonlinear model predictive control of large-scale systems. Part I: Description of the method. Automatisierungstechnik 50(01), 557–567 (2002)
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., Riedmiller, M.: Playing atari with deep reinforcement learning (2013)
Nonlinear optimization problem solver ipopt. https://github.com/coin-or/Ipopt. Accessed 10 Feb 2023
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal policy optimization algorithms (2017)
Raffin, A., Hill, A., Gleave, A., Kanervisto, A., Ernestus, M., Dormann, N.: Stable-baselines3: reliable reinforcement learning implementations. J. Mach. Learn. Res. 22(268), 1–8 (2021)
Acknowledgments
The research was partially supported by the Government program (contract #123021700055-6) and partially supported by RFBR and Moscow city Government (project number 21-31-70005).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Nunuparov, A., Syrykh, N. (2024). Nonlinear Model Predictive Control and Reinforcement Learning for Capsule-Type Robot with an Opposing Spring. In: Youssef, E.S.E., Tokhi, M.O., Silva, M.F., Rincon, L.M. (eds) Synergetic Cooperation between Robots and Humans. CLAWAR 2023. Lecture Notes in Networks and Systems, vol 811. Springer, Cham. https://doi.org/10.1007/978-3-031-47272-5_12
Download citation
DOI: https://doi.org/10.1007/978-3-031-47272-5_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-47271-8
Online ISBN: 978-3-031-47272-5
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)