Skip to main content
SpringerLink
Log in

A real-time seam tracking system based on extraction and restoration model in robotic GMAW

  • Original Article
  • Published:
The International Journal of Advanced Manufacturing Technology Aims and scope Submit manuscript

Abstract

Seam tracking technology is an important part of the intelligent welding field. In this paper, a laser vision-based real-time seam tracking system was built. The system consists of a self-developed laser vision sensor, a six-axis robot, a gas metal arc welding system, and an industrial computer. After building the system, the system calibration was performed. During the seam tracking, the arc light, spatter, and other welding noise have a negative impact on the image processing algorithm to extract the weld feature points, and even lead to system drift and algorithm failure. To this end, a two-stage extraction and restoration model (ERM) was proposed for processing real-time welding images to improve the robustness and accuracy of the seam tracking system. In the ERM, the region of interest was first detected and extracted by the YOLOv5s model, then the extracted images were restored by the conditional generation adversarial network. After using the ERM model, a series of image processing was performed to obtain the coordinates of the weld feature points. The total time consumed by the algorithm is 37 ms per frame on average, which meets the real-time requirement. Moreover, the experimental results show that the seam tracking system based on the ERM can achieve real-time tracking for different types of planar V-bevel welds, and the average error is 0.21 mm, which meets the requirements for actual welding.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

Data availability

Not applicable.

References

  1. Wang B, Hu S, Sun L, Freiheit T (2020) Intelligent welding system technologies: State-of-the-art review and perspectives. J Manuf Syst 56:373–391. https://doi.org/10.1016/j.jmsy.2020.06.020

    Article  Google Scholar 

  2. Xu F, Xu Y, Zhang H, Chen S (2022) Application of sensing technology in intelligent robotic arc welding: a review. J Manuf Process 79:854–880. https://doi.org/10.1016/j.jmapro.2022.05.029

    Article  Google Scholar 

  3. Lei T, Rong Y, Wang H, Huang Y, Li M (2020) A review of vision-aided robotic welding. Comput Ind 123:103326. https://doi.org/10.1016/j.compind.2020.103326

    Article  Google Scholar 

  4. Yang L, Li E, Long T, Fan J, Liang Z (2018) A high-speed seam extraction method based on the novel structured-light sensor for arc welding robot: a review. IEEE Sens J 18:8631–8641. https://doi.org/10.1109/JSEN.2018.2867581

    Article  Google Scholar 

  5. Rout A, Deepak BBVL, Biswal BB (2019) Advances in weld seam tracking techniques for robotic welding: a review. Robot Comput Integr Manuf 56:12–37. https://doi.org/10.1016/j.rcim.2018.08.003

    Article  Google Scholar 

  6. Lei T, Huang Y, Shao W, Liu W, Rong Y (2020) A tactual weld seam tracking method in super narrow gap of thick plates. Robot Comput Integr Manuf 62:101864. https://doi.org/10.1016/j.rcim.2019.101864

    Article  Google Scholar 

  7. Mahajan A, Figueroa F (1997) Intelligent seam tracking using ultrasonic sensors for robotic welding. Robotica 15:275–281. https://doi.org/10.1017/S0263574797000313

    Article  Google Scholar 

  8. Le J, Zhang H, Chen X (2018) Realization of rectangular fillet weld tracking based on rotating arc sensors and analysis of experimental results in gas metal arc welding. Robot Comput Integr Manuf 49:263–276. https://doi.org/10.1016/j.rcim.2017.06.004

    Article  Google Scholar 

  9. Xu Y, Fang G, Lv N, Chen S, Zou J (2015) Computer vision technology for seam tracking in robotic GTAW and GMAW. Robot Comput Integr Manuf 32:25–36. https://doi.org/10.1016/j.rcim.2014.09.002

    Article  Google Scholar 

  10. Shah H, Sulaiman M, Shukor A, Kamis Z, Ab Rahman A (2018) Butt welding joints recognition and location identification by using local thresholding. Robot Comput Integr Manuf 51:181–188. https://doi.org/10.1016/j.rcim.2017.12.007

    Article  Google Scholar 

  11. Mao Y, Xu G (2022) A real-time method for detecting weld deviation of corrugated plate fillet weld by laser vision sensor. Optik 260:168786. https://doi.org/10.1016/j.ijleo.2022.168786

    Article  Google Scholar 

  12. Zhao X, Zhang Y, Wang H, Liu Y, Zhang B, Hu S (2022) Research on trajectory recognition and control technology of real-time tracking welding. Sensors 22:8546. https://doi.org/10.3390/s22218546

    Article  Google Scholar 

  13. Deng L, Lei T, Wu C, Liu Y, Cao S, Zhao S (2023) A weld seam feature real-time extraction method of three typical welds based on target detection. Measurement 207:112424. https://doi.org/10.1016/j.measurement.2022.112424

    Article  Google Scholar 

  14. Zou Y, Chen X, Gong G, Li J (2018) A seam tracking system based on a laser vision sensor. Measurement 127:489–500. https://doi.org/10.1016/j.measurement.2018.06.020

    Article  Google Scholar 

  15. Shao W, Huang Y, Zhang Y (2018) A novel weld seam detection method for space weld seam of narrow butt joint in laser welding. Opts Laser Technol 99:39–51. https://doi.org/10.1016/j.optlastec.2017.09.037

    Article  Google Scholar 

  16. Kiddee P, Fang Z, Tan M (2016) An automated weld seam tracking system for thick plate using cross mark structured light. Int J Adv Manuf Technol 87:3589–3603. https://doi.org/10.1007/s00170-016-8729-7

    Article  Google Scholar 

  17. Xu P, Xu G, Tang X, Yao S (2008) A visual seam tracking system for robotic arc welding. Int J Adv Manuf Technol 37:70–75. https://doi.org/10.1007/s00170-007-0939-6

    Article  Google Scholar 

  18. Zhang G, Zhang Y, Tuo S, Hou Z, Yang W, Xu Z, Wu Y, Yuan H, Kyoosik S (2021) A novel seam tracking technique with a four-step method and experimental investigation of robotic welding oriented to complex welding seam. Sensors 21:3067. https://doi.org/10.3390/s21093067

    Article  Google Scholar 

  19. Xiao R, Xu Y, Hou Z, Chen C, Chen S (2021) A feature extraction algorithm based on improved Snake model for multi-pass seam tracking in robotic arc welding. J Manuf Process 72:48–60. https://doi.org/10.1016/j.jmapro.2021.10.005

    Article  Google Scholar 

  20. Fan J, Jing F, Yang L, Long T, Tan M (2019) A precise seam tracking method for narrow butt seams based on structured light vision sensor. Opts Laser Technol 109:616–626. https://doi.org/10.1016/j.optlastec.2018.08.047

    Article  Google Scholar 

  21. Chen Y, Shi Y, Cui Y, Chen X (2021) Narrow gap deviation detection in keyhole TIG welding using image processing method based on Mask-RCNN model. Int J Adv Manuf Technol 112:2015–2025. https://doi.org/10.1007/s00170-020-06466-5

    Article  Google Scholar 

  22. Zou Y, Chen T, Chen X, Li J (2022) Robotic seam tracking system combining convolution filter and deep reinforcement learning. Mech Syst Signal Process 165:108372–108085. https://doi.org/10.1016/j.ymssp.2021.108372

    Article  Google Scholar 

  23. Lin Z, Shi Y, Wang Z, Li B, Chen X (2023) Intelligent seam tracking of an ultranarrow gap during K-TIG welding: a hybrid CNN and adaptive ROI operation algorithm. IEEE Trans Instrum Meas 72:1–14. https://doi.org/10.1109/TIM.2022.3230475

    Article  Google Scholar 

  24. Liu C, Shen J, Hu S, Wu D, Zhang C, Yang H (2022) Seam tracking system based on laser vision and CGAN for robotic multi-layer and multi-pass MAG welding. Eng Appl Artif Intell 116:105377. https://doi.org/10.1016/j.engappai.2022.105377

    Article  Google Scholar 

  25. Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 779–788. https://doi.org/10.1109/CVPR.2016.91

    Chapter  Google Scholar 

  26. GitHub (2021) YOLOV5-Master. https://github.com/ultralytics/yolov5.git/. Accessed 14 Oct 2022

  27. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2020) Generative adversarial nets. Commun ACM 63(11):139–144. https://doi.org/10.1145/3422622

    Article  Google Scholar 

  28. Mirza M, Osindero S (2014) Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784v1. https://doi.org/10.48550/arXiv.1411.1784

  29. Zhang Z (2000) A flexible new technique for camera calibration. IEEE Trans Pattern Anal Mach Intell 22:1330–1334. https://doi.org/10.1109/34.888718

    Article  Google Scholar 

  30. Fan J, Jing F, Fang Z, Liang Z (2016) A simple calibration method of structured light plane parameters for welding robots. In: Proceedings of the 35th Chinese Control Conference, pp 6127–6132. https://doi.org/10.1109/ChiCC.2016.7554318

    Chapter  Google Scholar 

  31. Tsai R, Lenz R (1989) A new technique for fully autonomous and efficient 3d robotics hand/eye calibration. IEEE Trans Robot Autom 5:345–358. https://doi.org/10.1109/70.34770

    Article  Google Scholar 

  32. Wang C, Liao H, Wu Y, Chen P, Hsieh J, Yeh I (2020) CSPNet: a new backbone that can enhance learning capability of CNN. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 390–391. https://doi.org/10.1109/CVPRW50498.2020.00203

    Chapter  Google Scholar 

  33. He K, Zhang X, Ren S, Sun J (2015) Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans Pattern Anal Machine Intell 37(9):1904–1916. https://doi.org/10.1109/TPAMI.2015.2389824

    Article  Google Scholar 

  34. Liu S, Qi L, Qin H, Shi J, Jia J (2018) Path aggregation network for instance segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 8759–8768. https://doi.org/10.1109/CVPR.2018.00913

    Chapter  Google Scholar 

Download references

Funding

This work was supported by the National Natural Science Foundation of China (No. 52275338) and the Innovation Platform and Talent Specialization of Jilin Provincial Department of Science and Technology (No. 20230508039RC).

Author information

Authors and Affiliations

  1. Key Laboratory of Automobile Materials, School of Materials Science and Engineering, Jilin University, Changchun, 130025, China

    Xiaohui Zhao, Bin Yang, Ziwei Li, Yongchang Liang, Yupeng Chi, Yunhao Chen & Hao Wang

Authors
  1. Xiaohui Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bin Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ziwei Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yongchang Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yupeng Chi
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yunhao Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Hao Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Xiaohui Zhao: validation original draft, conceptualization, project administration, and supervision. Bin Yang: background research and investigation, analysis, writing and validation original manuscript, methodology, software, editing. Ziwei Li: editing, validation original draft, supervision. Yongchang Liang: validation original draft, software, investigation. Yupeng Chi: investigation, data curation. Yunhao Chen: investigation. Hao Wang: validation original draft, conceptualization, supervision. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Hao Wang.

Ethics declarations

Ethics approval

The authors confirm that they have abided to the publication ethics and state that this work is original and has not been used for publication anywhere before.

Consent to participate

The authors are willing to participate in journal promotions and updates.

Consent for publication

The authors give consent to the journal regarding the publication of this work.

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhao, X., Yang, B., Li, Z. et al. A real-time seam tracking system based on extraction and restoration model in robotic GMAW. Int J Adv Manuf Technol 130, 3805–3818 (2024). https://doi.org/10.1007/s00170-024-12959-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00170-024-12959-4

Keywords

Search

Navigation