Abstract
The inspection and maintenance of power distribution network are crucial for efficiently delivering electricity to consumers. Due to the high voltage of power distribution network lines, manual live-line operations are difficult, risky, and inefficient. This paper researches a Power Distribution Network Live-line Operation Robot (PDLOR) with autonomous tool assembly capabilities to replace humans in various high-risk electrical maintenance tasks. To address the challenges of tool assembly in dynamic and unstructured work environments for PDLOR, we propose a framework consisting of deep visual-guided coarse localization and prior knowledge and fuzzy logic driven deep deterministic policy gradient (PKFD-DPG) high-precision assembly algorithm. First, we propose a multiscale identification and localization network based on YOLOv5, which enables the peg-hole close quickly and reduces ineffective exploration. Second, we design a main-auxiliary combined reward system, where the main-line reward uses the hindsight experience replay mechanism, and the auxiliary reward is based on fuzzy logic inference mechanism, addressing ineffective exploration and sparse reward in the learning process. In addition, we validate the effectiveness and advantages of the proposed algorithm through simulations and physical experiments, and also compare its performance with other assembly algorithms. The experimental results show that, for single-tool assembly tasks, the success rate of PKFD-DPG is 15.2% higher than the DDPG with functionized reward functions and 51.7% higher than the PD force control method; for multip-tools assembly tasks, the success rate of PKFD-DPG method is 17% and 53.4% higher than the other methods.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Availability of data and materials
Not applicable
Code availability
The codes and datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.
References
Alhassan, A.B., Zhang, X., Shen, H., Xu, H.: Power transmission line inspection robots: A review, trends and challenges for future research. Int. J. Electr. Power Energy Syst. 118, 105862 (2020)
Jenssen, R., Roverso, D., et al.: Automatic autonomous vision-based power line inspection: A review of current status and the potential role of deep learning. Int. J. Electr. Power Energy Syst. 99, 107–120 (2018)
Chen, Y., Wang, Y., Tang, X., Wu, K., Wu, S., Guo, R., Feng, Y., Dong, E.: Intelligent power distribution live-line operation robot systems based on stereo camera. High Voltage (2023)
Jiang, Y., Huang, Z., Yang, B., Yang, W.: A review of robotic assembly strategies for the full operation procedure: planning, execution and evaluation. Robot. Comput. Integr. Manuf. 78, 102366 (2022)
Kotsiopoulos, T., Sarigiannidis, P., Ioannidis, D., Tzovaras, D.: Machine learning and deep learning in smart manufacturing: The smart grid paradigm. Comput. Sci. Rev. 40, 100341 (2021)
Li, Z., YaHao, W., Run, Y., Shaolei, W., Rui, G., Dong, E.: An efficiently convergent deep reinforcement learning-based trajectory planning method for manipulators in dynamic environments. J. Intell. Robot. Syst. 107(4) (2023)
Chhatpar, S.R., Branicky, M.S.: Search strategies for peg-in-hole assemblies with position uncertainty. In: Proceedings 2001 IEEE/RSJ International Conference on Intelligent Robots and Systems. Expanding the Societal Role of Robotics in the the Next Millennium (Cat. No. 01CH37180), vol. 3, pp. 1465–1470. IEEE (2001)
Kang, H., Zang, Y., Wang, X., Chen, Y.: Uncertainty-driven spiral trajectory for robotic peg-in-hole assembly. IEEE Robotics and Automation Letters (2022)
Jasim, I.F., Plapper, P.W., Voos, H.: Position identification in force-guided robotic peg-in-hole assembly tasks. Procedia Cirp 23, 217–222 (2014)
Chen, F., Cannella, F., Huang, J., Sasaki, H., Fukuda, T.: A study on error recovery search strategies of electronic connector mating for robotic fault-tolerant assembly. J. Intell. Robot. Syst. 81(2), 257–271 (2016)
Park, H., Park, J., Lee, D.-H., Park, J.-H., Bae, J.-H.: Compliant peg-in-hole assembly using partial spiral force trajectory with tilted peg posture. IEEE Robot. Autom. Lett. 5(3), 4447–4454 (2020)
Abu-Dakka, F.J., Nemec, B., Kramberger, A., Buch, A.G., Krüger, N., Ude, A.: Solving peg-in-hole tasks by human demonstration and exception strategies. Industrial Robot: An International Journal (2014)
Park, H., Park, J., Lee, D.-H., Park, J.-H., Baeg, M.-H., Bae, J.-H.: Compliance-based robotic peg-in-hole assembly strategy without force feedback. IEEE Trans. Ind. Electron. 64(8), 6299–6309 (2017)
Jiang, T., Cui, H., Cheng, X., Tian, W.: A measurement method for robot peg-in-hole prealignment based on combined two-level visual sensors. IEEE Trans. Instrum. Meas. 70, 1–12 (2020)
Xu, J., Liu, K., Pei, Y., Yang, C., Cheng, Y., Liu, Z.: A noncontact control strategy for circular peg-in-hole assembly guided by the 6-dof robot based on hybrid vision. IEEE Trans. Instrum. Meas. 71, 1–15 (2022)
Lu, B.-S., Chen, T.-I., Lee, H.-Y., Hsu, W.H.: Cfvs: Coarse-to-fine visual servoing for 6-dof object-agnostic peg-in-hole assembly. In: 2023 IEEE International Conference on Robotics and Automation (ICRA), pp. 12402–12408. IEEE (2023)
Yasutomi, A.Y., Ichiwara, H., Ito, H., Mori, H., Ogata, T.: Visual spatial attention and proprioceptive data-driven reinforcement learning for robust peg-in-hole task under variable conditions. IEEE Robot. Autom. Lett. 8(3), 1834–1841 (2023)
Wang, J., Jiang, Y., Lin, S., Kong, F.: Geometric model-based joint angle selection criterion for force parameter identification & decoupling control method of position and posture in shaft-hole assembly. In: 2021 IEEE 11th Annual International Conference on Cyber Technology in Automation, Control, and Intelligent Systems (CYBER), pp. 7–12. IEEE (2021)
Kim, M.-C., Choi, H., Piao, J., Kim, E.-S., Park, J.-O., Kim, C.-S.: Remotely manipulated peg-in-hole task conducted by cable-driven parallel robots. IEEE/ASME Trans. Mechatron. 27(5), 3953–3963 (2022)
Tang, X., Shang, W., Hu, J., Zhang, F., Zhang, X.: Error state probability-based compliance control for peg-in-hole assembly. IEEE Trans. Autom. Sci. Eng. (2023)
Zhao, Y., Gao, F., Zhao, Y., Chen, Z.: Peg-in-hole assembly based on six-legged robots with visual detecting and force sensing. Sensors 20(10), 2861 (2020)
Chen, Z., Xie, S., Zhang, X.: Position/force visual-sensing-based robotic sheet-like peg-in-hole assembly. IEEE Trans. Instrum. Meas. 71, 1–11 (2021)
Lee, D.-H., Choi, M.-S., Park, H., Jang, G.-R., Park, J.-H., Bae, J.-H.: Peg-in-hole assembly with dual-arm robot and dexterous robot hands. IEEE Robot. Autom. Lett. 7(4), 8566–8573 (2022)
Higuera, C., Ortiz, J., Qi, H., Pineda, L., Boots, B., Mukadam, M.: Perceiving extrinsic contacts from touch improves learning insertion policies. arXiv:2309.16652 (2023)
Van der Merwe, M., Wi, Y., Berenson, D., Fazeli, N.: Integrated object deformation and contact patch estimation from visuo-tactile feedback. arXiv:2305.14470 (2023)
Fan, Y., Luo, J., Tomizuka, M.: A learning framework for high precision industrial assembly. In: 2019 International conference on robotics and automation (ICRA), pp. 811–817. IEEE (2019)
Leyendecker, L., Schmitz, M., Zhou, H.A., Samsonov, V., Rittstieg, M., Lütticke, D.: Deep reinforcement learning for robotic control in high-dexterity assembly tasks-a reward curriculum approach. In: 2021 Fifth IEEE International Conference on Robotic Computing (IRC), pp. 35–42. IEEE (2021)
Petrovic, O., Schäper, L., Roggendorf, S., Storms, S., Brecher, C.: Sim2real deep reinforcement learning of compliance-based robotic assembly operations. In: 2022 26th International Conference on Methods and Models in Automation and Robotics (MMAR), pp. 300–305. IEEE (2022)
Inoue, T., De Magistris, G., Munawar, A., Yokoya, T., Tachibana, R.: Deep reinforcement learning for high precision assembly tasks. In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 819–825. IEEE (2017)
Xie, L., Yu, H., Zhao, Y., Zhang, H., Zhou, Z., Wang, M., Wang, Y., Xiong, R.: Learning to fill the seam by vision: Sub-millimeter peg-in-hole on unseen shapes in real world. In: 2022 International Conference on Robotics and Automation (ICRA), pp. 2982–2988. IEEE (2022)
Schoettler, G., Nair, A., Luo, J., Bahl, S., Ojea, J.A., Solowjow, E., Levine, S.: Deep reinforcement learning for industrial insertion tasks with visual inputs and natural rewards. In: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 5548–5555. IEEE (2020)
Wang, Y., Zhao, L., Zhang, Q., Zhou, R., Wu, L., Ma, J., Zhang, B., Zhang, Y.: Alignment method of combined perception for peg-in-hole assembly with deep reinforcement learning. J. Sensors 2021, 1–12 (2021)
Lämmle, A., Tenbrock, P., Bálint, B., Nägele, F., Kraus, W., Váncza, J., Huber, M.F.: Simulation-based learning of the peg-in-hole process using robot-skills. In: 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 9340–9346. IEEE (2022)
Beltran-Hernandez, C.C., Petit, D., Ramirez-Alpizar, I.G., Nishi, T., Kikuchi, S., Matsubara, T., Harada, K.: Learning force control for contact-rich manipulation tasks with rigid position-controlled robots. IEEE Robot. Autom. Lett. 5(4), 5709–5716 (2020)
Ren, S., He, K., Girshick, R., Sun, J.: Faster r-cnn: Towards real-time object detection with region proposal networks. Adv. Neural Inf Process Syst. 28 (2015)
He, K., Gkioxari, G., Dollár, P., Girshick, R.: Mask r-cnn. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2961–2969. (2017)
Redmon, J., Farhadi, A.: Yolov3: An incremental improvement. (2018) arXiv:1804.02767
Bochkovskiy, A., Wang, C.-Y., Liao, H.-Y.M.: YOLOv4: Optimal Speed and Accuracy of Object Detection (2020)
Jiang, P., Ergu, D., Liu, F., Cai, Y., Ma, B.: A review of yolo algorithm developments. Procedia Comput. Sci. 199, 1066–1073 (2022)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT press, ??? (2018)
Eschmann, J.: Reward function design in reinforcement learning. Reinforcement Learning Algorithms: Analysis and Applications, 25–33 (2021)
Gupta, A., Pacchiano, A., Zhai, Y., Kakade, S., Levine, S.: Unpacking reward shaping: Understanding the benefits of reward engineering on sample complexity. Adv. Neural Inf. Process. Syst. 35, 15281–15295 (2022)
Andrychowicz, M., Wolski, F., Ray, A., Schneider, J., Fong, R., Welinder, P., McGrew, B., Tobin, J., Pieter Abbeel, O., Zaremba, W.: Hindsight experience replay. Adv. Neural Inf. Process. Syst. 30 (2017)
Magdalena, L.: Fuzzy rule-based systems. Springer handbook of computational intelligence, 203–218 (2015)
Acknowledgements
This work was supported by the National Key R &D Program of China (Grant number:2018YFB1307400) and State Grid Anhui Science and Technology Project.
Funding
This work was supported by the National Key R &D Program of China (Grant numbers[2018YFB1307400]).
Author information
Authors and Affiliations
Contributions
Li Zheng:Conceptualization, Methodology, Software, Validation, Data curation, Writing- Original draft preparation, Investigation. Jiajun Ai: Conceptualization, Data curation. Xuming Tang: Resources, Data curation. Shaolei Wu: Resources, Data curation. Sheng Chen: Resources, Data curation. Rui Guo: Resources, Data curation. Erbao Dong: Supervision, Writing- Reviewing and Editing. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Conflict of interest/Competing interests
The authors have no relevant financial or non-financial interests to disclose.
Ethical standard
Not applicable
Consent to participate
Not applicable
Consent for publication
Not applicable
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Zheng, L., Ai, J., Wang, Y. et al. Deep Visual-guided and Deep Reinforcement Learning Algorithm Based for Multip-Peg-in-Hole Assembly Task of Power Distribution Live-line Operation Robot. J Intell Robot Syst 110, 77 (2024). https://doi.org/10.1007/s10846-024-02079-2
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10846-024-02079-2