Skip to main content
SpringerLink
Log in

A Behavior-based Adaptive Dynamic Programming Method for Multiple Mobile Manipulators Coordination Control

  • Regular Papers
  • Robot and Applications
  • Published:
International Journal of Control, Automation and Systems Aims and scope Submit manuscript

Abstract

In this work, a behavior-based adaptive dynamic programming (BADP) method is proposed for coordination control of unmanned ground vehicle-manipulator systems (UGVMs). Through a null-space-based behavioral control (NSBC) framework, the multi-objective coordination control is transformed into a single-objective tracking control at the mission layer. Since cost functions and control constraints are simplified at control layer, the complexity of controller design is reduced. Then, an identifier-actor-critic reinforcement learning algorithm framework is introduced to learn the optimal control policy by balancing the control performance and consumption. Simulation results show that control costs are reduced around 13.5% per sampling period compared to existing multiple objective control methods. Finally, the BADP method is experimentally validated using four real UGVMs.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. G. B. Dai and Y. C. Liu, “Distributed coordination and cooperation control for networked mobile manipulators,” IEEE Transactions on Industrial Electronics, vol. 64, no. 6, pp. 5065–5074, June 2017.

    Article  Google Scholar 

  2. G. B. Dai and Y. C. Liu, “Distributed coordination and cooperation control for networked mobile manipulators,” IEEE Transactions on Industrial Electronics, vol. 64, no. 6, pp. 5065–5074, June 2017.

    Article  Google Scholar 

  3. A. Brahmi, M. Saad, G. Gauthier, W. H. Zhu, and J. Ghommam, “Adaptive control of multiple mobile manipulators transporting a rigid object,” International Journal of Control, Automation and Systems, vol. 15, no. 4, pp. 1779–1789, September 2017.

    Article  Google Scholar 

  4. T. Zhao, Y. Y. Liu, Z. J. Li, C. Y. Su, and Y. Feng, “Adaptive control and optimization of mobile manipulation subject to input saturation and switching constraints,” IEEE Transactions on Automation Science and Engineering, vol. 16, no. 4, pp. 1543–1555, December 2018.

    Article  Google Scholar 

  5. W. Li and R. Xiong, “Dynamical obstacle avoidance of task-constrained mobile manipulation using model predictive control,” IEEE Access, vol. 7, no. 1, pp. 88301–88311, July 2019.

    Article  Google Scholar 

  6. J. Liao, F. Huang, and Z. Chen, “Optimization-based motion planning of mobile manipulator with high degree of kinematic redundancy,” International Journal of Intelligent Robotics and Applications, vol. 3, no. 2, pp. 115–130, April 2019.

    Article  Google Scholar 

  7. U. Ahmad, Y. J. Pan, and H. H. Shen, “Robust control design for teleoperation of multiple mobile manipulators under time delays,” International Journal of Robust and Nonlinear Control, vol. 30, no. 16, pp. 6454–6472, May 2019.

    Article  MathSciNet  Google Scholar 

  8. C. Wu, H. Fang, and X. L. Zeng, “Distributed object transport of mobile manipulators with optimal manipulable coordination,” 37th Chinese Control Conference (CCC), pp. 7043–7048, 2018.

  9. Y. Q. Zhang, Z. H. Deng, and Y. G. Hong, “Distributed optimal coordination for multiple heterogeneous Euler-Lagrangian systems,” Automatica, vol. 79, no. 1, pp. 207–213, May 2017.

    Article  MathSciNet  MATH  Google Scholar 

  10. J. Alonso-Mora, S. Baker, and D. Rus, “Multi-robot formation control and object transport in dynamic environments via constrained optimization,” The International Journal of Robotics Research, vol. 36, no. 9, pp. 1000–1021, September 2017.

    Article  Google Scholar 

  11. J. Chen, S. X. Kai, “Cooperative transportation control of multiple mobile manipulators through distributed optimization,” Science China Information Sciences, vol. 61, no. 12, pp. 1–17, November 2018.

    Article  MathSciNet  Google Scholar 

  12. Y. J. Du, B. L. Ma, and D. Y. Meng, “Distributed control for signed networks of nonlinear agents,” International Journal of Control, Automation, and Systems, vol. 18, no. 2, pp. 271–281, July 2019.

    Article  Google Scholar 

  13. D. Liu, S. Xue, Bo Zhao, and Q. L. Wei, “Adaptive dynamic programming for control: a survey and recent advances,” IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 51, no. 1, pp. 142–160, January 2021.

    Article  Google Scholar 

  14. H. Y. Dong, X. W. Zhao, and Bo Zhao, “Optimal tracking control for uncertain nonlinear systems with prescribed performance via critic-only ADP,” IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 52, no. 1, pp. 561–573, January 2022.

    Article  Google Scholar 

  15. Z. F. Ye, Z. J. Li, C. Y. Su, and B. Huang, “Adaptive tracking control of a class of constrained Euler-Lagrange systems by factorization of dynamic mass matrix,” IEEE Transactions on Industrial Electronics, vol. 66, no. 10, pp. 7831–7840, October 2019.

    Article  Google Scholar 

  16. G. X. Wen, C. L. P. Chen, J. Feng, and N. Zhou, “Optimized multi-agent formation control based on an identifier-actor-critic reinforcement learning algorithm,” IEEE Transactions on Fuzzy Systems, vol. 26, no. 5, pp. 2719–2731, October 2018.

    Article  Google Scholar 

  17. G. X. Wen, C. L. P. Chen, and B. Li, “Optimized formation control using simplified reinforcement learning for a class of multiagent systems with unknown dynamics,” IEEE Transactions on Fuzzy Systems, vol. 67, no. 9, pp. 7879–7888, September 2020.

    Google Scholar 

  18. G. X. Wen, C. L. P. Chen, and Y. J. Liu, “Formation control with obstacle avoidance for a class of stochastic multiagent systems,” IEEE Transactions on Industrial Electronics, vol. 65, no. 7, pp. 5847–5855, July 2018.

    Article  Google Scholar 

  19. Z. H. Pan, D. Wang, H. B. Deng and K. W. Li, “A virtual spring method for the multi-robot path planning and formation control,” International Journal of Control, Automation and Systems, vol. 17, no. 5, pp. 1272–1282, December 2018.

    Article  Google Scholar 

  20. V. G. Lopez and F. L. Lewis, “Dynamic multiobjective control for continuous-time systems using reinforcement learning,” IEEE Transactions on Automatic Control, vol. 64, no. 7, pp. 2869–2874, July 2019.

    Article  MathSciNet  MATH  Google Scholar 

  21. M. Mazouchi, Y. L. Yang, and H. Modares, “Data-driven dynamic multiobjective optimal control: an aspiration-satisfying reinforcement learning approach,” IEEE Transactions on Neural Networks and Learning Systems, vol. 1, no. 1, pp. 1–11, April 2021.

    Google Scholar 

  22. S. Khankalantary, I. Izadi, and F. Sheikholeslam, “Robust ADP-based solution of a class of nonlinear multi-agent systems with input saturation and collision avoidance constraints,” ISA transactions, vol. 107, no. 1, pp. 52–62, July 2020.

    Article  Google Scholar 

  23. M. L. Greene, P. Deptula, S. Nivison and W. E. Dixon, “Sparse learning-based approximate dynamic programming with barrier constraints,” IEEE Control Systems Letters, vol. 4, no. 3, pp. 743–748, July 2020.

    Article  MathSciNet  Google Scholar 

  24. Y. L. Yang, K. G. Vamvoudakis, H. Modares, W. He, Y. X. Yin, and D. C. Donald, “Safety-aware reinforcement learning framework with an actor-critic-barrier structure,” 2019 American Control Conference (ACC), pp. 2352–2358, 2018.

  25. X. J. Lan, L. Lei, and Y. J. Wang, “ADP-based intelligent decentralized control for multi-agent systems moving in obstacle environment,” IEEE Access, vol. 7, no. 1, pp. 59624–59630, May 2019.

    Article  Google Scholar 

  26. K. Baizid, G. Giglio, F. Pierri, M. A. Trujillo, G. Antonelli, F. Caccavale, A. Viguria, S. Chiaverini, and A. Ollero, “Behavioral control of unmanned aerial vehicle manipulator systems,” Autonomous Robots, vol. 14, no. 1, pp. 1203–1220, July 2017.

    Article  Google Scholar 

  27. D. W. Wang and L. C. Boon, “Modeling and analysis of skidding and slipping in wheeled mobile robots: Control design perspective,” IEEE Transactions on Robotics, vol. 24, no. 3, pp. 676–687, June 2008.

    Article  Google Scholar 

  28. J. Huang, N. Zhou, and M. Cao, “Adaptive fuzzy behavioral control of second-order autonomous agents with prioritized missions: Theory and experiments,” IEEE Transactions on Industrial Electronics, vol. 66, no. 12, pp. 9612–9622, December 2019.

    Article  Google Scholar 

  29. C. Ott, A. Dietrich, and A. Albu-Schäffer, “Prioritized multi-task compliance control of redundant manipulators,” Automatica, vol. 53, no. 1, pp. 416–423, February 2015.

    Article  MathSciNet  MATH  Google Scholar 

  30. G. Antonelli, “Stability analysis for prioritized closed-loop inverse kinematic algorithms for redundant robotic systems,” IEEE Transactions on Robotics, vol. 25, no. 5, pp. 985–994, April 2009.

    Article  Google Scholar 

  31. Y. T. Chen, Z. Y. Zhang, and J. Huang, “Dynamic Task Priority Planning for Null-Space Behavioral Control of Multi-Agent Systems,” IEEE Access, vol. 8, no. 1, pp. 2169–3536, August 2020.

    Google Scholar 

  32. Z. Y. Zhang, Z. B. Mo, Y. T. Chen, and J. Huang, “Reinforcement Learning Behavioral Control for Nonlinear Autonomous System,” IEEE/CAA Journal of Automatica Sinica, vol. 9, no. 9, pp. 1561–1573, August 2022.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. College of Electrical Engineering and Automation, Fuzhou University, Fuzhou, 350108, China

    Zhenyi Zhang, Jianfei Chen, Zhibin Mo, Yutao Chen & Jie Huang

  2. 5G+ Industrial Internet Institute, Fuzhou University, Fuzhou, 350108, China

    Zhenyi Zhang, Jianfei Chen, Zhibin Mo, Yutao Chen & Jie Huang

Authors
  1. Zhenyi Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jianfei Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhibin Mo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yutao Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jie Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jie Huang.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work was supported in part by the National Natural Science Foundation of China under Grants 61603094.

Zhenyi Zhang received his B.E. degree in mechanical and electronic engineering, and an M.E. degree in mechanical engineering from Zhejiang Sci-Tech University, Hangzhou, China, in 2016 and 2019. He is now a Ph.D. student at Fuzhou University, China. His research interests include intelligent robot ethology and multiagent learning systems.

Jianfei Chen received his B.E. degree in measurement and control technology and instrumentation from Qingdao University of Science and Technology, Shandong, China, in 2020. He is now an M.E. student at Fuzhou University, Fuzhou, China. His research interests include aerial-ground cooperative systems optimization and multi-agent systems.

Zhibin Mo received his B.E. degree in Shantou University, Shantou, China, in 2018. He is pursuing an M.E. degree in Fuzhou University, Fuzhou, China. His research interests include multi-agent reinforcement learning and human-robot coordination systems.

Yutao Chen received his B.E. degree in automation from Hunan University, China, in 2012, a master’s degree from the School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University, China, in 2014, and a Ph.D. degree from the Department of Information Engineering, University of Padova, Italy, in 2018. From 2019 to 2020, he was a Post-doctoral Researcher with the Department of Electrical Engineering, Eindhoven University of Technology, The Netherlands. He is currently an Assistant Professor with the College of Electrical Engineering and Automation, Fuzhou University, China. His research interests include model predictive control algorithms and unmanned intelligent systems and applications.

Jie Huang received his B.E. degree in electrical engineering and automation, an M.E. degree in control engineering from Fuzhou University, China, and a Ph.D. degree in control science and engineering from Beijing Institute of Technology, Beijing, China, in 2005, 2010, and 2015, respectively. From 2005 to 2015, he was a lecturer with Fujian Institute of Education, Fuzhou, China. From 2014 to 2016, he was postdoctoral researcher with the Faculty of Mathematics and Natural Sciences, University of Groningen, the Netherlands. From 2016 to 2018, he held lecturer appointments with the Faculty of Science and Engineering, University of Groningen, the Netherlands. He is currently a full professor of robotic and control with the College of Electrical Engineering and Automation, Fuzhou University, China and the director of the 5G+ Industrial Internet Institute, Fuzhou University, China. He is the vice-president of the Fujian Automation Association, Fujian Province, China. His research interests include autonomous robots, complex network dynamics, and multi-agent systems.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, Z., Chen, J., Mo, Z. et al. A Behavior-based Adaptive Dynamic Programming Method for Multiple Mobile Manipulators Coordination Control. Int. J. Control Autom. Syst. 21, 3022–3035 (2023). https://doi.org/10.1007/s12555-021-0904-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12555-021-0904-4

Keywords

Search

Navigation