Abstract
In this article, a robot skills learning framework is developed, which considers both motion modeling and execution. In order to enable the robot to learn skills from demonstrations, a learning method called dynamic movement primitives (DMPs) is introduced to model motion. A staged teaching strategy is integrated into DMPs frameworks to enhance the generality such that the complicated tasks can be also performed for multi-joint manipulators. The DMP connection method is used to make an accurate and smooth transition in position and velocity space to connect complex motion sequences. In addition, motions are categorized into different goals and durations. It is worth mentioning that an adaptive neural networks (NNs) control method is proposed to achieve highly accurate trajectory tracking and to ensure the performance of action execution, which is beneficial to the improvement of reliability of the skills learning system. The experiment test on the Baxter robot verifies the effectiveness of the proposed method.
References
Q. Fu, X. Q. Wang, Y. Zou, W. He. A miniature video stabilization system for? Apping-wing aerial vehicles. Guidance, Navigation and Control, vol. 2, no. 1, Article number 2250001, 2022. DOI: https://doi.org/10.1142/S2737480722500017.
H. Qiao, M. Wang, J. H. Su, S. X. Jia, R. Li. The concept of “attractive region in environment” and its application in high-precision tasks with low-precision syatems. IEEE/ASME Transactions on Mechatronics, vol. 20, no. 5, pp. 2311–2327, 2015. DOI: https://doi.org/10.1109/TMECH.2014.2375638.
Z. J. Li, B. Huang, A. Ajoudani, C. G. Yang, C. Y. Su, A. Bicchi. Asymmetric bimanual control of dual-arm exoskeletons for human-cooperative manipulations. IEEE Transactions on Robotics, vol. 34, no. 1, pp. 264–271, 2018. DOI: https://doi.org/10.1109/TRO.2017.2765334.
L. Li, Z. Y. Chen, Y. B. Wang, X. D. Zhang, N. F. Wang. Robust task-space tracking for free-floating space manipulators by cerebellar model articulation controller. Assembly Automation, vol. 30, no. 1, pp. 26–33, 2010. DOI: https://doi.org/10.1108/AA-11-2017-143.
M. Bortolini, M. Faccio, F. G. Galizia, M. Gamberi, F. Pilati. Design, engineering and testing of an innovative adaptive automation assembly system. Assembly Automation, vol. 40, no. 3, pp. 531–540, 2020. DOI: https://doi.org/10.1108/AA-06-2019-0103.
X. B. Yu, W. He, Y. A. Li, C. Q. Xue, Y. K. Sun, Y. Wang. Adaptive NN impedance control for an SEA-driven robot. Science China Information Sciences, vol. 63, no. 5, Article number 159207, 2020. DOI: https://doi.org/10.1007/s11432-018-9631-7.
C. Zeng, C. G. Yang, Z. P. Chen. Bio-inspired robotic impedance adaptation for human-robot collaborative tasks. Science China Information Sciences, vol. 63, no. 7, Article number 170201, 2020. DOI: https://doi.org/10.1007/s11432-019-2748-x.
L. Kunze, N. Hawes, T. Duckett, M. Hanheide, T. Krajník. Artificial intelligence for long-term robot autonomy: A survey. IEEE Robotics and Automation Letters, vol. 3, no. 4, pp. 4023–4030, 2018. DOI: https://doi.org/10.1109/LRA.2018.2860628.
Y. Gigras, K. Gupta. Artificial intelligence in robot path planning. International Journal of Soft Computing and Engineering, vol. 2, no. 2, pp. 2231–2307, 2012.
W. He, Z. J. Li, C. L. P. Chen. A survey of human-centered intelligent robots: Issues and challenges. IEEE/CAA Journal of Automatica Sinica, vol. 4, no. 4, pp. 602–600, 2017. DOI: https://doi.org/10.1109/JAS.2017.7510604.
Y. Y. Qi, J. C. Liu, J. Z. Yu. Dynamic modeling and hybrid fireworks algorithm-based path planning of an amphibious robot. Guidance, Navigation and Control, vol. 2, no. 1, Article number 2250002, 2022. DOI: https://doi.org/10.1142/S2737480722500029.
Z. J. Li, X. Li, Q. J. Li, H. Su, Z. Kan, W. He. Human-in-the-loop control of soft exosuits using impedance learning on different terrains. IEEE Transactions on Robotics, vol. 38, no. 5, pp. 2070–2003, 2022. DOI: https://doi.org/10.1109/TRO.2022.3160052.
Y. M. Jiang, Y. N. Wang, Z. Q. Miao, J. Na, Z. J. Zhao, C. G. Yang. Composite-learning-based adaptive neural control for dual-arm robots with relative motion. IEEE Transactions on Neural Networks and Learning Systems, vol. 33, no. 3, pp. 1010–1021, 2022. DOI: https://doi.org/10.1109/TNNLS.2020.3037795.
S. Schaal. Is imitation learning the route to humanoid robots? Trends in Cognitive Sciences, vol. 3, no. 6, pp. 233–242, 1999. DOI: https://doi.org/10.1016/S1364-6613(99)01327-3.
M. Asif Rana, M. Mukadam, S. Reza Ahmadzadeh, S. Chernova, B. Boots. Learning generalizable robot skills from demonstrations in cluttered environments. In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, IEEE, Madrid, Spain, pp. 4655–4660, 2018. DOI: https://doi.org/10.1109/IROS.2018.8593624.
C. G. Yang, C. Zeng, C. Fang, W. He, Z. J. Li. A DMPs-based framework for robot learning and generalization of humanlike variable impedance skills. IEEE/ASME Transactions on Mechatronics, vol. 23, no. 3, pp. 1103–1203, 2018. DOI: https://doi.org/10.1109/TMECH.2018.2817589.
S. Calinon, A. Pistillo, D. G. Caldwell. Encoding the time and space constraints of a task in explicit-duration hidden Markov model. In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, IEEE, San Francisco, USA, pp. 3413–3418, 2011. DOI: https://doi.org/10.1109/IROS.2011.6094418.
L. Rozo, S. Calinon, D. G. Caldwell, P. Jiménez, C. Torras. Learning physical collaborative robot behaviors from human demonstrations. IEEE Transactions on Robotics, vol. 32, no. 3, pp. 513–527, 2016. DOI: https://doi.org/10.1109/TRO.2016.2540623.
A. J. Ijspeert, J. Nakanishi, H. Hoffmann, P. Pastor, S. Schaal. Dynamical movement primitives: Learning attractor models for motor behaviors. Neural Computation, vol. 25, no. 2, pp. 328–373, 2013. DOI: https://doi.org/10.1162/NECO_a_00393.
A. Ude, A. Gams, T. Asfour, J. Morimoto. Task-specific generalization of discrete and periodic dynamic movement primitives. IEEE Transactions on Robotics, vol. 26, no. 5, pp. 800–815, 2010. DOI: https://doi.org/10.1109/TRO.2010.2065430.
P. Pastor, H. Hoffmann, T. Asfour, S. Schaal. Learning and generalization of motor skills by learning from demonstration. In Proceedings of IEEE International Conference on Robotics and Automation, Kobe, Japan, pp. 763–768, 2009. DOI: https://doi.org/10.1109/ROBOT.2009.5152385.
T. Petrič, A. Gams, L. Colasanto, A. J. Ijspeert, A. Ude. Accelerated sensorimotor learning of compliant movement primitives. IEEE Transactions on Robotics, vol. 34, no. 6, pp. 1636–1642, 2018. DOI: https://doi.org/10.1109/TRO.2018.2861921.
Y. Wu, R. H. Wang, L. F. D’Haro, R. E. Banchs, K. P. Tee. Multi-modal robot apprenticeship: Imitation learning using linearly decayed DMP+ in a human-robot dialogue system. In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, IEEE, Madrid, Spain, pp. 8585–8588, 2018. DOI: https://doi.org/10.1109/IROS.2018.8593634.
C. G. Yang, C. Z. Chen, W. He, R. X. Cui, Z. J. Li. Robot learning system based on adaptive neural control and dynamic movement primitives. IEEE Transactions on Neural Networks and Learning Systems, vol. 30, no. 3, pp. 777–787, 2019. DOI: https://doi.org/10.1109/TNNLS.2018.2852711.
A. Pervez, D. Lee. Learning task-parameterized dynamic movement primitives using mixture of GMMs. Intelligent Service Robotics, vol. 11, no. 1, pp. 61–78, 2018. DOI: https://doi.org/10.1007/s11370-017-0235-8.
Z. J. Li, T. Zhao, F. Chen, Y. B. Hu, C. Y. Su, T. Fukuda. Reinforcement learning of manipulation and grasping using dynamical movement primitives for a humanoidlike mobile manipulator. IEEE/ASME Transactions on Mechatronics, vol. 23, no. 1, pp. 121–131, 2018. DOI: https://doi.org/10.1109/TMECH.2017.2717461.
T. Kulvicius, K. J. Ning, M. Tamosiunaite, F. Wörgötter. Joining movement sequences: Modified dynamic movement primitives for robotics applications exemplified on handwriting. IEEE Transactions on Robotics, vol. 28, no. 1, pp. 145–157, 2012. DOI: https://doi.org/10.1109/TRO.2011.2163863.
B. E. Perk, J. J. E. Slotine. Motion primitives for robotic flight control. [Online], Available: https://arxiv.org/abs/cs/0609140, 2006.
W. Wang, J. J. E. Slotine. On partial contraction analysis for coupled nonlinear oscillators. Biological Cybernetics, vol. 92, no. 1, pp. 38–53, 2005. DOI: https://doi.org/10.1007/s00422-004-0527-x.
Z. J. Li, Z. C. Huang, W. He, C. Y. Su. Adaptive impedance control for an upper limb robotic exoskeleton using biological signals. IEEE Transactions on Industrial Electronics, vol. 64, no. 2, pp. 1664–1674, 2017. DOI: https://doi.org/10.1109/TIE.2016.2538741.
W. Wu, Y. M. Li, S. C. Tong. Neural network output-feedback consensus fault-tolerant control for nonlinear multiagent systems with intermittent actuator faults. IEEE Transactions on Neural Networks and Learning Systems, to be published. DOI: https://doi.org/10.1109/TNNLS.2021.3117364.
L. H. Kong, W. He, C. G. Yang, Z. J. Li, C. Y. Sun. Adaptive fuzzy control for coordinated multiple robots with constraint using impedance learning. IEEE Transactions on Cybernetics, vol. 49, no. 8, pp. 3052–3063, 2019. DOI: https://doi.org/10.1109/TCYB.2018.2838573.
J. Y. Li, Z. H. Zhan, J. Zhang. Evolutionary computation for expensive optimization: A survey. Machine Intelligence Research, vol. 19, no. 1, pp. 3–23, 2022. DOI: https://doi.org/10.1007/s11633-022-1317-4.
B. B. Jia, M. L. Zhang. Multi-dimensional classification via selective feature augmentation. Machine Intelligence Research, vol. 19, no. 1, pp. 38–51, 2022. DOI: https://doi.org/10.1007/s11633-022-1316-5.
Y. L. Yang, K. G. Vamvoudakis, H. Modares, Y. X. Yin, D. C. Wunsch. Safe intermittent reinforcement learning with static and dynamic event generators. IEEE Transactions on Neural Networks and Learning Systems, vol. 31, no. 12, pp. 5441–5455, 2020. DOI: https://doi.org/10.1109/TNNLS.2020.2967871.
H. Y. Li, Y. Wu, M. Chen, R. Q. Lu. Adaptive multigradient recursive reinforcement learning event-triggered tracking control for multiagent systems. IEEE Transactions on Neural Networks and Learning Systems, to be published. DOI: https://doi.org/10.1109/TNNLS.2021.3090570.
Y. Liu, X. B. Chen, Y. L. Wu, H. Cai, H. Yokoi. Adaptive neural network control of a flexible spacecraft subject to input nonlinearity and asymmetric output constraint. IEEE Transactions on Neural Networks and Learning Systems, to be published. DOI: https://doi.org/10.1109/TNNLS.2021.3072907.
P. van Cuong, W. Y. Nan. Adaptive trajectory tracking neural network control with robust compensator for robot manipulators. Neural Computing and Applications, vol. 27, no. 2, pp. 525–536, 2016. DOI: https://doi.org/10.1007/s00521-015-1873-4.
C. X. Liu, G. L. Wen, Z. J. Zhao, R. Sedaghati. Neural-network-based sliding-mode control of an uncertain robot using dynamic model approximated switching gain. IEEE Transactions on Cybernetics, vol. 51, no. 5, pp. 2339–2346, 2021. DOI: https://doi.org/10.1109/TCYB.2020.2978003.
Y. Ren, Z. J. Zhao, C. L. Zhang, Q. M. Yang, K. S. Hong. Adaptive neural-network boundary control for a flexible manipulator with input constraints and model uncertainties. IEEE Transactions on Cybernetics, vol. 51, no. 10, pp. 4796–4807, 2021. DOI: https://doi.org/10.1109/TCYB.2020.3021069.
M. Chen, S. S. Ge, B. V. E. How. Robust adaptive neural network control for a class of uncertain mimo nonlinear systems with input nonlinearities. IEEE Transactions on Neural Networks, vol. 21, no. 5, pp. 796–812, 2010. DOI: https://doi.org/10.1109/TNN.2010.2042611.
L. H. Kong, W. He, Y. T. Dong, L. Cheng, C. G. Yang, Z. J. Li. Asymmetric bounded neural control for an uncertain robot by state feedback and output feedback. IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 51, no. 3, pp. 1735–1746, 2021. DOI: https://doi.org/10.1109/TSMC.2019.2901277.
H. W. Lin, B. Zhao, D. R. Liu, C. Alippi. Data-based fault tolerant control for affine nonlinear systems through particle swarm optimized neural networks. IEEE/CAA Journal of Automatica Sinica, vol. 7, no. 4, pp. 954–964, 2020. DOI: https://doi.org/10.1109/JAS.2020.1003225.
A. Chaouki, A. El Abed. Finite-time and fixed-time synchronization of inertial neural networks with mixed delays. Journal of Systems Science and Complexity, vol. 34, no. 1, pp. 206–235, 2021. DOI: https://doi.org/10.1007/s11424-020-9029-8.
H. Y. Li, S. Y. Zhao, W. He, R. Q. Lu. Adaptive finite-time tracking control of full state constrained nonlinear systems with dead-zone. Automatica, vol. 100, pp. 99–107, 2019. DOI: https://doi.org/10.1016/j.automatica.2018.10.030.
Q. T. Zhao, J. Sun, Y. Q. Bai. Dynamic event-triggered control for nonlinear systems: A small-gain approach. Journal of Systems Science and Complexity, vol. 33, no. 4, pp. 930–943, 2020. DOI: https://doi.org/10.1007/s11424-020-9210-0.
Z. J. Liu, Z. J. Han, Z. J. Zhao, W. He. Modeling and adaptive control for a spatial flexible spacecraft with unknown actuator failures. Science China Information Sciences, vol. 64, no. 5, Article number 152208, 2021. DOI: https://doi.org/10.1007/s11432-020-3109-x.
L. H. Kong, W. He, Z. J. Liu, X. B. Yu, C. Silvestre. Adaptive tracking control with global performance for output-constrained MIMO nonlinear systems. IEEE Transactions on Automatic Control, to be published. DOI: https://doi.org/10.1109/TAC.2022.3201258.
Acknowledgements
This work was supported by National Natural Science Foundation of China (Nos. 62225304, 92148204 and 62061160371), in part by National Key Research and Development Program of China (Nos. 2021ZD0114503 and 2019YFB1703600), Beijing Top Discipline for Artificial Intelligence Science and Engineering, University of Science and Technology Beijing, and the Beijing Natural Science Foundation (No. JQ20026).
Author information
Authors and Affiliations
Corresponding author
Additional information
Ling-Huan Kong received the B. Eng. degree in automation from College of Engineering, Qufu Normal University, China in 2016 and the M. Eng. degree in control engineering from School of Automation Engineering, University of Electronic Science and Technology of China, China in 2019. He is currently a Ph. D. degree candidate in control science and engineering with School of Automation and Electrical Engineering, University of Science and Technology Beijing, China.
His research interests include robotics, neural network control, and adaptive control.
Wei He received the B. Eng. and M. Eng. degrees in automation science and engineering from the South China University of Technology, China in 2006 and 2008, respectively, and the Ph. D. degree in electrical and computer engineering from the National University of Singapore, Singapore in 2011. He is currently working as a full professor with School of Intelligence Science and Technology and Institute of Artificial Intelligence, University of Science and Technology Beijing, China. He has co-authored three books published in Springer and over 100 international journal and conference papers. He was a recipient of the IEEE Systems, Man, and Cybernetics Society Andrew P. Sage Best Transactions Paper Award. He was awarded a Newton Advanced Fellowship from the Royal Society, UK. He is serving as the Chair of IEEE Systems, Man, and Cybernetics Society Beijing Capital Region Chapter. Since 2018, he has been the Chair of Technical Committee on Autonomous Bionic Robotic Aircraft, and IEEE Systems, Man and Cybernetics Society. He is a highly cited researcher by Clarivate Analytics from 2019 to 2021. He is serving as an Associate Editor for IEEE Transactions on Robotics, IEEE Transactions on Neural Networks and Learning Systems, IEEE Transactions on Control Systems Technology, IEEE Transactions on Systems, Man, and Cybernetics: Systems, Science China Information Sciences, IEEE/CAA Journal of Automatica Sinica, Assembly Automation, and Machine Intelligence Research.
His research interests include robotics, distributed parameter systems, and intelligent control systems.
Wen-Shi Chen received the B. Eng. and M. Eng. degrees in automation from School of Automation and Electrical Engineering, University of Science and Technology Beijing (USTB), China in 2018 and 2021, respectively.
His research interests include robotics, neural network control, and dynamic movement primitives.
Hui Zhang received the B. Sc, M. Sc and Ph. D. degrees in pattern recognition and intelligent system from Hunan University, China in 2004, 2007 and 2012, respectively. He is currently a professor with School of Robotics, Hunan University, China. He was a visiting scholar with the CVSS Laboratory, Department of Electrical and Computer Engineering, University of Windsor, Canada.
His research interests include machine vision, sparse representation, and image processing.
Yao-Nan Wang received the B. Sc. degree in computer engineering from the East China University of Science and Technology, China in 1981, and the M. Sc. and Ph. D. degrees in control engineering from Hunan University, China in 1990 and 1994, respectively. He was a postdoctoral research fellow with National University of Defense Technology, China from 1994 to 1995, a Senior Humboldt Fellow in Germany from 1998 to 2000, and a visiting professor with University of Bremen, Germany from 2001 to 2004. He has been a professor with Hunan University, China since 1995. He has been a member of China Engineering Academy since 2019.
His research interests include robot control, intelligent control and information processing, industrial process control, and image processing.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.
The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
To view a copy of this licence, visit https://doi.org/creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Kong, LH., He, W., Chen, WS. et al. Dynamic Movement Primitives Based Robot Skills Learning. Mach. Intell. Res. (2023). https://doi.org/10.1007/s11633-022-1346-z
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s11633-022-1346-z
Keywords
- Dynamic movement primitives (DMPs)
- trajectory tracking control
- robot learning from demonstrations
- neural networks (NNs)
- adaptive control