Skip to main content
SpringerLink
Log in

Vibration Reduction Control of In-Pipe Intelligent Isolation Plugging Tool Based on Deep Reinforcement Learning

  • Regular Paper
  • Published:
International Journal of Precision Engineering and Manufacturing-Green Technology Aims and scope Submit manuscript

Abstract

Compared with traditional plugging methods, the in-pipe intelligent isolation plugging tool (IPT) is advantageous in safety and work efficiency. However, during the plugging process, the flow field around the IPT changes drastically, resulting in vortex-induced vibration and potential failure of the plugging operation. In this study, three foldable spoilers were designed at the tail of the IPT to optimize the flow field. The vibration of the IPT can be alleviated by adjusting the angles of the spoilers. A vibration reduction control system of the IPT was designed based on deep reinforcement learning. First, we conducted an experiment for vibration reduction system. Second, a nonlinear model of the pressure difference based on experimental data was established. Then, a multi-agent self-learning system based on the deep Q-network (DQN) was designed, and the optimal actions were selected in each agent to adjust the spoiler angles during the plugging process. Finally, a controller based on fuzzy reinforcement learning was proposed to flip the spoilers to the optimized angles. The results show that the vibration reduction control system of the IPT reduced the pressure difference by an average of 28.32%, which indicates the stability of the plugging process and a successful reduction of the IPT vibration.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

References

  1. Tubb, R. (2014). Worldwide construction report. Pipeline and Gas Journal, 241, 1–6.

    Google Scholar 

  2. Mirshamsi, M., & Rafeeyan, M. (2015). Dynamic analysis and simulation of long pig in gas pipeline. Journal of Natural Gas Science and Engineering, 23, 294–303.

    Article  Google Scholar 

  3. Ma, M., Zhao, H., Su, X., et al. (2014). The current development status and prospect of oil and gas pipeline plugging and emergency repair technology. China Petroleum Machinery, 42(06), 109–118.

    Google Scholar 

  4. Zhang, S. M., Mei, X. T., Wang, G. C., et al. (2014). Oil/gas pipeline maintenance and emergency repair method and its progress. Oil & Gas Storage And Transportation, 33(11), 1180–1186.

    Google Scholar 

  5. Zhang, K., Zhang, S. M., & Ding, Q. X. (2018). Research on the effects of Packer’s hardness on self-sealing in pipeline isolation tool. Oil Field Equipment, 47(02), 8–12.

    Google Scholar 

  6. Zhang, K., Zhang, S. M., Zhu, X. X., et al. (2017). Study on the new pipeline maintenance technology based on the smart plugging technique. China Petroleum Machinery, 45(01), 114–119.

    Google Scholar 

  7. Selden RA. (2009). Innovative solution for emergency repair of a deep-water riser. In: The offshore technology conference, 4–7 May, Houston, Texas, USA. https://doi.org/10.4043/20154-MS.

  8. Rush A.S. (2009). Proceedings of Offshore Technology Conference. Houston: Society of Petroleum Engineers.

  9. He, T., Wang, W. B., Zhao, H., et al. (2013). Fluent-based numerical simulation of the flow field around the plugging tools. Oil & Gas Storage And Transportation, 32(06), 615–619.

    Google Scholar 

  10. Shen, K. (2016). The development of feeding and connecting mechanism of spherical double-plugging plug in Φ711 mm pipeline. China University of Petroleum.

    Google Scholar 

  11. Quan, Y. (2017). Design of Φ508 spherical double plugging for oil and gas pipeline. China University of Petroleum.

    Google Scholar 

  12. Li, Z. B., You, J., Cheng, H. H., et al. (2019). Structural design and plugging performance of 508 mm single-section double-seal internal plugging device. Oil & Gas Storage And Transportation, 38(09), 1065–1071.

    Google Scholar 

  13. Li, Z. B., Hu, T. H., Fu, D. R., et al. (2016). Influence of the profile angle of slip in pipeline plugging device on the anchoring performance. Oil & Gas Storage And Transportation, 35(10), 1060–1065.

    Google Scholar 

  14. Li, Z. B., Zhang, J., Jing, R., et al. (2019). The effect of the rubber tube length of in-pipe plug on the plugging performance. Oil & Gas Storage And Transportation, 38(07), 810–815.

    Google Scholar 

  15. Zhao, H., & Hu, H. R. (2016). Flow field analysis on internal plug at different plugging states. Oil & Gas Storage And Transportation, 35(10), 1060–1065.

    Google Scholar 

  16. Zhao, H., & Hu, H. R. (2017). Optimal design of a pipe isolation plugging tool using a computational fluid dynamics simulation with response surface methodology and a modified genetic algorithm. Advances in Mechanical Engineering, 9(10), 1–12.

    Article  Google Scholar 

  17. Hu, H. R., & Zhao, H. (2016). Optimization of end face structure of smart plug. China Petroleum Machinery, 44(05), 71–76.

    Google Scholar 

  18. Zhao, H., Hu, H. R., & He, T. (2018). Simulation on the distribution laws of flow field around the in-pipe intelligent plug of different end faces. Oil & Gas Storage And Transportation, 37(04), 413–420.

    Google Scholar 

  19. Ma, M., Zhao, H., & Su, X. (2014). DAMS-based kinetic research on the locking mechanism of pipeline plugging device. China Petroleum Machinery, 42(04), 101–105.

    Google Scholar 

  20. Rahman, M. M., Rashid, S. M. H., & Hossain, M. M. (2018). Implementation of Q learning and deep Q network for controlling a self -balancing robot model. Robotics and Biomimetics, 5(1), 1–6.

    Article  Google Scholar 

  21. Cheng, Y., Huang, Y. X., Pang, B., et al. (2018). ThermalNet: A deep reinforcement learning-based combustion optimization system for coal-fired boiler. Engineering Applications of Artificial Intelligence, 74, 303–311.

    Article  Google Scholar 

  22. Hao, G. F., Fu, Z., & Zheng, H. (2020). Vehicle speed tracking control with a robotic driver based on deep Q-network. Machinery & Electronics, 38(09), 50-53+64.

    Google Scholar 

  23. Zhou, S. L., Yang, X., Liu, K. Z., et al. (2020). COLREGs-compliant method for ship collision avoidance based on deep reinforcement learning. Navigation of China, 43(03), 27-32+46.

    Google Scholar 

  24. Ricardo, M., Me, J., Rui, A., et al. (2020). Regenerative braking system modeling by fuzzy Q-Learning. Engineering Applications of Artificial Intelligence, 93, 103712.

    Article  Google Scholar 

  25. Yan, A., Chen, Z., Dong, C. Y., et al. (2021). Attitude balance control of two-wheeled robot based on fuzzy reinforcement learning. Systems Engineering and Electronics. http://kns.cnki.net/kcms/detail/11.2422.TN.20201106.1644.008.html. Retrieved 26 Feb 2021.

  26. Du, Z. J., Wang, W., Yan, Z. Y., et al. (2017). A physical human-robot interaction algorithm based on fuzzy reinforcement learning for minimally invasive surgery manipulator. Robot, 39(03), 363–370.

    Google Scholar 

  27. Ren, L. W., Ban, X. J., Wu, F., et al. (2019). Fuzzy learning controller design of 2-DOF flight attitude simulator. Electric Machines and Control, 23(11), 127–134.

    Google Scholar 

  28. Miao, X. Y., Zhao, H., & Wang, C. J. (2020). Design and optimization of vibration reduction structure of intelligent pluggers inside pipe. China Petroleum Machinery, 48(05), 99–106.

    Google Scholar 

  29. Wu, T. T., & Zhao, H. (2019). An energy-saving and velocity-tracking control design for the pipe isolation tool. Advances in Mechanical Engineering, 11(4), 1–13.

    Article  Google Scholar 

  30. Zhang, K., Ding, Q. X., Liu, S. H., et al. (2019). Research on pressure fluctuation phenomenon using the smart isolation tool in subsea pipeline maintenance operation. Proceedings of the Institution of Mechanical Engineers, 233(2), 643–652.

    Google Scholar 

  31. Zhu, X. X., Wang, H., & Li, F. Q. (2017). Research on the automatically trajectory control of a spherical isolation plug in subsea pipeline. Ocean Engineering, 141, 101–107.

    Article  Google Scholar 

  32. Miao, X. Y., & Zhao, H. (2020). Optimal design of vibration structure of intelligent plugging isolation tool based on Monte Carlo algorithm. Oil Field Equipment, 49(06), 46–54.

    Google Scholar 

  33. Zhao, H., Zhao, Y. X., & Ye, Z. H. (2016). Interactions between the fluid and an isolation tool in a pipe: Laboratory experiments and numerical simulation. Petroleum Science, 13(04), 746–759.

    Article  Google Scholar 

  34. Huang, S. C., & Dao, T. P. (2016). Design and computational optimization of a flexure-based XY positioning platform using FEA-based response surface methodology. International Journal of Precision Engineering and Manufacturing, 17(08), 1035–1048.

    Article  Google Scholar 

  35. Prakash, C. S., Reetimukta, B., Ramesh, C. P., et al. (2020). Automatic generation control of diverse energy source-based multiarea power system under deep Q-network-based fuzzy-T2 controller. Energy Sources. https://doi.org/10.1080/15567036.2020.1809568

    Article  Google Scholar 

  36. Wu, T.T., Zhao, H., Gao, B.X., et al. (2021). Energy-saving for a velocity control system of a pipe isolation tool based on a reinforcement learning method. International Journal of Precision Engineering and Manufacturing-Green Technology, 1–16.

  37. Li, W., Ye, J. Y., Cui, Y. D., et al. (2021). A speedy reinforcement learning-based energy management strategy for fuel cell hybrid vehicles considering fuel cell system lifetime. International Journal of Precision Engineering and Manufacturing-Green Technology, 1–14.

  38. Valarezo, A.E., Rivera, L. P., Park, N. (2020). Natural object manipulation using anthropomorphic robotic hand through deep reinforcement learning and deep grasping probability network. Applied Intelligence, 51(2).

  39. Hu, H. R. (2017). Study on Simulation and Control Optimization in the Plugging Process of SmartPlug. China University of Petroleum.

    Google Scholar 

  40. Goharimanesh, M., Mehrkish, A., & Janabi-Sharifi, F. (2020). A fuzzy reinforcement learning approach for continuum robot control. Journal of Intelligent & Robotic Systems. Advance Online Publication. https://doi.org/10.1007/s10846-020-01237-6

    Article  Google Scholar 

  41. Chen, C. H., Jeng, S. Y., & Lin, C. J. (2020). Mobile robot wall-following control using fuzzy logic controller with improved differential search and reinforcement learning. Mathematics, 8, 1254.

    Article  Google Scholar 

  42. Hoang, V.D., Xuan, D.T., Hoai, V.A., et al. (2020). Optimization-based fuzzy energy management strategy for PEM fuel cell/battery/supercapacitor hybrid construction excavator. International Journal of Precision Engineering and Manufacturing-Green Technology, 1–19.

Download references

Acknowledgements

This work was financially supported by the National Natural Science Foundation of China (Grant No. 51575528), the Science Foundation of China University of Petroleum, Beijing (No.2462020XKJS01).

Author information

Author notes

  1. Xingyuan Miao and Hong Zhao contributed equally to this work as corresponding author.

Authors and Affiliations

  1. College of Mechanical and Transportation Engineering, China University of Petroleum, Beijing, 102249, China

    Xingyuan Miao, Hong Zhao, Boxuan Gao, Tingting Wu & Yanguang Hou

Authors
  1. Xingyuan Miao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hong Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Boxuan Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tingting Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yanguang Hou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Xingyuan Miao or Hong Zhao.

Ethics declarations

Conflict of interest

On behalf of all authors, the corresponding author states that there is no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Miao, X., Zhao, H., Gao, B. et al. Vibration Reduction Control of In-Pipe Intelligent Isolation Plugging Tool Based on Deep Reinforcement Learning. Int. J. of Precis. Eng. and Manuf.-Green Tech. 9, 1477–1491 (2022). https://doi.org/10.1007/s40684-021-00405-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40684-021-00405-9

Keywords

Search

Navigation