Skip to main content

Advertisement

SpringerLink
Log in

Decentralized approximated optimal control for modular robot manipulations with physical human–robot interaction: a cooperative game-based strategy

  • Original Paper
  • Published:
Nonlinear Dynamics Aims and scope Submit manuscript

Abstract

Major challenges of controlling physical human–robot interaction (pHRI)-oriented modular robot manipulator (MRM) include performance optimization and solving the coupling effect between the human and the robot as well as MRM subsystems. In this paper, a cooperative game-based decentralized optimal control approach is developed for MRMs with pHRI. The joint torque feedback (JTF) technique is utilized to form the MRM dynamic model, then, the major objective of optimal control with pHRI is transformed into approximating Pareto equilibrium by adopting cooperative game governed between the human and the MRM that are regarded as players with different tasks and optimization criteria in interaction process. On the basis of adaptive dynamic programming (ADP) algorithm, the decentralized approximate optimal control strategy with pHRI is developed by a critic neural network (NN)-based cooperative game manner for solving the coupled Hamilton–Jacobian–Bellman (HJB) equation. The position tracking error under pHRI is verified to be ultimately uniformly bounded (UUB). Experiment results have been presented, which exhibit the superiority of the proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Data availability

The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.

References

  1. Bednarczyk, M., Omran, H., Bayle, B.: EMG-based variable impedance control with passivity guarantees for collaborative robotics. IEEE Robotics Autom. Lett. 7(2), 4307–4312 (2022)

    Google Scholar 

  2. Villani, V., Pini, F., Leali, F., et al.: Survey on human-robot collaboration in industrial settings: safety, intuitive interfaces and applications. Mechatronics 55, 248–266 (2018)

    Google Scholar 

  3. Bogue, R.: Rehabilitation robots. Ind. Robot An Int. J. 45(3), 301–306 (2018)

    Google Scholar 

  4. Weber, L., Stein, J.: The use of robots in stroke rehabilitation: A narrative review. NeuroRehabilitation 43(1), 99–110 (2018)

    Google Scholar 

  5. Wang, Q., Liu, D., Carmichael, M., et al.: Computational model of robot trust in human co-worker for physical human–robot collaboration. IEEE Robotics Autom. Lett. 7(2), 3146–3153 (2022)

    Google Scholar 

  6. Cappello, D., Mylvaganam, T.: Distributed differential games for control of multi-agent systems. IEEE Trans. Control Netw

  7. Jin, Z., Liu, A., Zhang, W., et al.: A learning based hierarchical control framework for human–robot collaboration. IEEE Trans. Autom. Sci. Eng

  8. Von Neumann, J., Morgenstern, O.: Theory of Games and Economic Behavior, 2nd edn. Princeton Univ, Princeton, NJ, USA (1947)

    Google Scholar 

  9. Nekouei, E., Nair, G.N., Alpcan, T., et al.: Sample complexity of solving non-cooperative games. IEEE T. Inform. Theory. 66(2), 1261–1280 (2020)

    MathSciNet  Google Scholar 

  10. Fudenberg, D., Tirole, J.: Game Theory. MIT Press, Cambridge (1991)

    Google Scholar 

  11. Kearns, M., Littman, M., Singh, S.: Graphical models for game theory. Proc. UA I, 253–260 (2001)

    Google Scholar 

  12. Qi, N., Huang, Z., Zhou, F., et al.: A task-driven sequential overlapping coalition formation game for resource allocation in heterogeneous UAV networks. IEEE Trans. Mob. Comput

  13. Xiao, W., Zhou, Q., Liu, Y., et al.: Distributed reinforcement learning containment control for multiple nonholonomic mobile robots. IEEE Trans. Circuits Syst. I Regul. Pap. 69(2), 896–907 (2022)

    Google Scholar 

  14. Li, Y., Tee, K., Yan, R., et al.: A framework of human–robot coordination based on game theory and policy iteration. IEEE Trans. Robotics 32(6), 1408–1418 (2016)

    Google Scholar 

  15. Stalford, H.: Criteria for Pareto-optimality in cooperative differential games. J. Optim. Theory Appl. 9(6), 391–398 (1972)

    MathSciNet  Google Scholar 

  16. Zhang, H., Jiang, H., Luo, Y., et al.: Data-driven optimal consensus control for discrete-time multi-agent systems with unknown dynamics using reinforcement learning method. IEEE Trans. Ind. Electron. 64(5), 4091–4100 (2017)

    Google Scholar 

  17. Song, R., Li, J., Lewis, F.: Robust optimal control for disturbed nonlinear zero-sum differential games based on single NN and least squares. IEEE Trans. Syst. Man. Cybern. 50(11), 4009–4019 (2020)

    Google Scholar 

  18. Mu, C., Wang, K., Ni, Z.: Adaptive learning and sampled-control for nonlinear game systems using dynamic event-triggering strategy. IEEE Trans. Neural Net. Learn. Syst

  19. Wang, D., Ha, M., Zhao, M.: The intelligent critic framework for advanced optimal control. Artif. Intell. Rev. 55, 1–22 (2022)

    Google Scholar 

  20. Ha, M., Wang, D., Liu, D.: A novel value iteration scheme with adjustable convergence rate. IEEE Trans. Neural Netw. Learn

  21. Wei, Q., Lu, J., Zhou, T., et al.: Event-triggered near-optimal control of discrete-time constrained nonlinear systems with application to a boiler-turbine system. IEEE Trans. Ind. Inform. 18(6), 3926–3935 (2022)

    Google Scholar 

  22. Gao, X., Bai, W., Li, T., et al.: Broad learning system-based adaptive optimal control design for dynamic positioning of marine vessels. Nonlinear Dyn. 105, 1593–1609 (2021)

    Google Scholar 

  23. Beuchat, P., Warrington, J., Lygeros, J.: Accelerated point-wise maximum approach to approximate dynamic programming. IEEE Trans. Autom. Control 67(1), 251–266 (2022)

    MathSciNet  Google Scholar 

  24. Adams, S., Cody, T., Beling, P.: A survey of inverse reinforcement learning. Artif. Intell. Rev. 55, 4307–4346 (2022)

    Google Scholar 

  25. Li, Z., Wu, L., Xu, Y., et al.: Multi-stage real-time operation of a multi-energy microgrid with electrical and thermal energy storage assets: a data-driven MPC-ADP approach. IEEE Trans. Smart Grid 13(1), 213–226 (2022)

    Google Scholar 

  26. Yang, R., Wang, D., Qiao, J.: Policy gradient adaptive critic design with dynamic prioritized experience replay for wastewater treatment process control. IEEE T. Ind. Inform. 18(5), 3150–3158 (2022)

    Google Scholar 

  27. Liu, Y., Zhang, H., Yu, R., et al.: Data-driven optimal tracking control for discrete-time systems with delays using adaptive dynamic programming. J. Frankl. Inst. 355(13), 5649–5666 (2018)

    MathSciNet  Google Scholar 

  28. Li, Y., Wei, C., An, T., et al.: Event-triggered-based cooperative game optimal tracking control for modular robot manipulator with constrained input. Nonlinear Dyn. 109, 2759–2779 (2022)

    Google Scholar 

  29. Yang, H., Hu, Q., Dong, H., et al.: ADP-based spacecraft attitude control under actuator misalignment and pointing constraints. IEEE Trans. Ind. Electron. 69(9), 9342–9352 (2022)

    Google Scholar 

  30. Huang, J., Zhang, Z., Cai, F., et al.: Optimized formation control for multi-agent systems based on adaptive dynamic programming without persistence of excitation. IEEE Control Syst. Lett. 6, 1412–1417 (2022)

    MathSciNet  Google Scholar 

  31. Dong, B., An, T., Zhou, F., et al.: Decentralized robust zero-sum neuro-optimal control for modular robot manipulators in contact with uncertain environments: theory and experimental verification. Nonlinear Dyn. 97, 503–524 (2019)

    Google Scholar 

  32. Tazi, K., Abbou, F., Abdi, F.: Multi-agent system for microgrids: design, optimization and performance. Artif. Intell. Rev. 53, 1233–1292 (2020)

    Google Scholar 

  33. Li, K., Li, Y.: Adaptive NN optimal consensus fault-tolerant control for stochastic nonlinear multiagent systems. IEEE Trans. Neural Netw. Learn

  34. Ma, B., Dong, B., Zhou, F., et al.: Adaptive dynamic programming-based fault-tolerant position-force control of constrained reconfigurable manipulators. IEEE Access 8, 183286–183299 (2020)

    Google Scholar 

  35. Han, K., Feng, J., Yao, Y.: An integrated data-driven Markov parameters sequence identification and adaptive dynamic programming method to design fault-tolerant optimal tracking control for completely unknown model systems. J. Frankl. Inst. 354(13), 5280–5301 (2017)

    MathSciNet  Google Scholar 

  36. Xue, S., Luo, B., Liu, D., et al.: Constrained event-triggered \({{H}_{\infty }}\) control based on adaptive dynamic programming with concurrent learning. IEEE Trans. Syst. Man. Cybern. 52(1), 357–369 (2022)

    Google Scholar 

  37. Liu, Y., Li, X.: Decentralized robust adaptive control of nonlinear systems with unmodeled dynamics. IEEE Trans. Autom. Control 47(5), 848–856 (2002)

    MathSciNet  Google Scholar 

  38. Yang, X., He, H.: Adaptive dynamic programming for decentralized stabilization of uncertain nonlinear large-scale systems with mismatched interconnections. IEEE Trans. Syst. Man. Cybern. 50(8), 2870–2882 (2020)

    Google Scholar 

  39. Zhou, Z., Xu, H.: Decentralized adaptive optimal tracking control for massive autonomous vehicle systems with heterogeneous dynamics: a stackelberg game. IEEE Trans. Neural Netw. Learn. 32(12), 5654–5663 (2021)

    MathSciNet  Google Scholar 

  40. Dong, B., Zhou, F., Liu, K., et al.: Decentralized robust optimal control for modular robot manipulators via critic-identifier structure-based adaptive dynamic programming. Neural Comput. Appl. 32, 3441–3458 (2020)

    Google Scholar 

  41. An, T., Wang, Y., Liu, G., et al.: Cooperative game-based approximate optimal control of modular robot manipulators for human–robot collaboration. IEEE Trans. Cybern. 53(7), 4691–4703 (2023)

    Google Scholar 

  42. Liu, G., Abdul, S., Goldenberg, A.A.: Distributed control of modular and reconfigurable robot with torque sensing. Robotica 26(1), 75–84 (2008)

    Google Scholar 

  43. Rahman, M., Ikeura, R., Mizutani, K.: Investigation of the impedance characteristic of human arm for development of robots to cooperate with humans. JSME Int. J. Ser. C 45(2), 510–518 (2002)

    Google Scholar 

  44. Yu, X., Li, Y., Zhang, S., et al.: Estimation of human impedance and motion intention for constrained human-robot interaction. Neurocomputing 390, 268–279 (2020)

    Google Scholar 

  45. Li, Y., Ge, S.: Human-robot collaboration based on motion intention estimation. IEEE-ASME Trans. Mech. 19(3), 1007–1014 (2014)

  46. Mu, C., Wang, K., Ni, Z., et al.: Cooperative differential game-based optimal control and its application to power systems. IEEE Trans. Ind. Inform. 16(8), 5169–5179 (2020)

    Google Scholar 

  47. Zhao, B., Wang, D., Shi, G., Liu, D., Li, Y.: Decentralized control for large-scale nonlinear systems with unknown mismatched interconnections via policy iteration. IEEE Trans. Syst Man Cybern. 48(10), 1725–1735 (2018)

    Google Scholar 

  48. Li, Y., Tee, K., Chan, W., et al.: Continuous role adaptation for human–robot shared control. IEEE Trans. Robotics 31(3), 672–681 (2017)

    Google Scholar 

  49. Vamvoudakis, K., Lewis, F.: Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem. Automatica 46, 878–888 (2010)

    MathSciNet  Google Scholar 

  50. Dong, B., An, T., Zhu, X., et al.: Zero-sum game-based neuro-optimal control of modular robot manipulators with uncertain disturbance using critic only policy iteration. Neurocomputing 450(2), 183–196 (2021)

    Google Scholar 

  51. Ma, B., Li, Y., An, T., et al.: Compensator-critic structure-based neuro-optimal control of modular robot manipulators with uncertain environmental contacts using non-zero-sum games. Knowl. Based Syst. 224(13), 107100 (2021)

    Google Scholar 

  52. Wang, D., Qiao, J., Cheng, L.: An approximate neuro-optimal solution of discounted guaranteed cost control design. IEEE Trans. Cybern. 52(1), 77–86 (2022)

    Google Scholar 

  53. Li, Q., Wang, Z., Wang, W., et al.: A human-centered comprehensive measure of take-over performance based on multiple objective metrics. IEEE Trans. Intell. Transp. Syst. 24(4), 4235–4250 (2023)

    Google Scholar 

  54. Li, Q., Su, Y., Wang, W., et al.: Latent hazard notification for highly automated driving: Expected safety benefits and driver behavioral adaptation. IEEE Trans. Intell. Transp. Syst. https://doi.org/10.1109/TITS.2023.3280955

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China (62173047), the Scientific Technological Development Plan Project in Jilin Province of China (YDZJ202201ZYTS508).

Funding

The authors have not disclosed any funding.

Author information

Authors and Affiliations

  1. Department of Control Science and Engineering, Changchun University of Technology, Changchun, 130012, China

    Tianjiao An, Xinye Zhu, Bing Ma & Bo Dong

  2. CAS Key Laboratory of On-Orbit Manufacturing and Integration for Space Optics System, Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun, 130033, China

    Mingchao Zhu

Authors
  1. Tianjiao An
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xinye Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bing Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mingchao Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Bo Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bo Dong.

Ethics declarations

Conflict of interest

The authors declared no potential conflicts of interest with respect to the research, authorship and publication of this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

An, T., Zhu, X., Ma, B. et al. Decentralized approximated optimal control for modular robot manipulations with physical human–robot interaction: a cooperative game-based strategy. Nonlinear Dyn 112, 7145–7158 (2024). https://doi.org/10.1007/s11071-024-09437-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11071-024-09437-7

Keywords

Search

Navigation