Skip to main content
SpringerLink
Log in

Deep learning-based fusion hole state recognition and width extraction for thin plate TIG welding

  • Research Paper
  • Published:
Welding in the World Aims and scope Submit manuscript

Abstract

In an automatic tungsten inert gas welding process of thin plates with a reserved gap, weld pool shape and fusion hole size are closely related to the weld quality. A vision-based weld area monitoring system is designed to simultaneously monitor the fusion hole from the front and back of the workpiece. Considering the location of fusion hole is not stable, an improved target detection network (YOLOV3) was proposed to localize the fusion hole. And ResNet-d is used as the backbone network in this model. The optimized YOLOv3 achieves an accuracy of 95.03% on mAP50 and a prediction speed of 49.43 frames per second (FPS). A corresponding image processing algorithm is designed based on the regional growth method, which can obtain clear edges and the width of the fusion hole. The algorithm’s accuracy is verified by compared with the extracted fusion hole widths from simultaneous images obtained from the backside.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25

Similar content being viewed by others

References

  1. Wang H, Kovacevic R (2002) On-line monitoring of the keyhole welding pool in variable polarity plasma arc welding. Proceeding Inst Mech Eng Part B J Eng Manuf 216:1265–1276

    Article  Google Scholar 

  2. Cui Y-X, Shi Y-H, Ning Q et al (2021) Investigation into keyhole-weld pool dynamic behaviors based on HDR vision sensing of real-time K-TIG welding process through a steel/glass sandwich. Adv Manuf 9:136–144. https://doi.org/10.1007/s40436-020-00335-w

    Article  CAS  Google Scholar 

  3. Guo Y, Gao J, Wu CS, Gui X (2019) Correlation between fusion hole morphology and weld penetration in TIG welding. Int J Adv Manuf Technol 101. https://doi.org/10.1007/s00170-018-3063-x

  4. Li C, Gao J, Cao Y et al (2019) Visual observation of fusion hole in thin plate TIG welding with a reserved gap. J Manuf Process 45:634–641. https://doi.org/10.1016/j.jmapro.2019.08.002

    Article  Google Scholar 

  5. Cao Y, Wang X, Yan X et al (2020) Prediction of fusion hole perforation based on arc characteristics of front image in backing welding. Materials 13:4706. https://doi.org/10.3390/ma13214706

    Article  CAS  Google Scholar 

  6. Luo M, Shin Y (2015) Estimation of keyhole geometry and prediction of welding defects during laser welding based on a vision system and a radial basis function neural network. Int J Adv Manuf Technol 81. https://doi.org/10.1007/s00170-015-7079-1

  7. Knaak C, Kröger M, Schulze F, et al (2021) Deep Learning and conventional machine learning for image-based in-situ fault detection during laser welding: a comparative study. https://doi.org/10.20944/preprints202105.0272.v1

  8. Xia C, Pan Z, Fei Z et al (2020) Vision based defects detection for keyhole TIG welding using deep learning with visual explanation. J Manuf Process 56:845–855. https://doi.org/10.1016/j.jmapro.2020.05.033

    Article  Google Scholar 

  9. Wu D, Huang Y, Zhang P et al (2020) Visual-Acoustic penetration recognition in variable polarity plasma arc welding process using hybrid deep learning approach. IEEE Access 8:120417–120428. https://doi.org/10.1109/ACCESS.2020.3005822

    Article  Google Scholar 

  10. Wang Y, Han J, Lu J et al (2020) TIG stainless steel molten pool contour detection and weld width prediction based on Res-Seg. Metals 10:1495. https://doi.org/10.3390/met10111495

    Article  CAS  Google Scholar 

  11. Yang L, Fan J, Liu Y et al (2021) Automatic detection and location of weld beads with deep convolutional neural networks. IEEE Trans Instrum Meas 70:1–12. https://doi.org/10.1109/TIM.2020.3026514

    Article  Google Scholar 

  12. Tianjiao L, Hong B (2020) A optimized YOLO method for object detection. In: 2020 16th International Conference on Computational Intelligence and Security (CIS). pp 30–34

  13. Kim H, Nam K, Oh S, Ki H (2021) Deep-learning-based real-time monitoring of full-penetration laser keyhole welding by using the synchronized coaxial observation method. J Manuf Process 68:1018–1030. https://doi.org/10.1016/j.jmapro.2021.06.029

    Article  Google Scholar 

  14. Girshick R (2015) Fast R-CNN. ArXiv E-Prints arXiv:1504.08083

  15. Ren S, He K, Girshick R, Sun J (2017) Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 39:1137–1149. https://doi.org/10.1109/TPAMI.2016.2577031

    Article  Google Scholar 

  16. Liu W, Anguelov D, Erhan D, et al (2016) SSD: Single Shot multibox detector. Comput Vis – ECCV 2016 21–37. https://doi.org/10.1007/978-3-319-46448-0_2

  17. Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: Unified, real-time object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 779–788

  18. Simonyan K, Zisserman A (2015) Very Deep convolutional networks for large-scale image recognition. arXiv:1409.1556. https://arxiv.org/abs/1409.1556

  19. Szegedy C, Vanhoucke V, Ioffe S, et al (2016) Rethinking the inception architecture for computer vision. 2818–2826. https://doi.org/10.1109/CVPR.2016.308

  20. He K, Zhang X, Ren S, Sun J (2015) Deep residual learning for image recognition. ArXiv E-Prints arXiv:1512.03385

  21. Jie H, Li S, Gang S, Albanie S (2020) Squeeze-and-excitation networks. IEEE Transactions on Pattern Analysis and Machine Intelligence 42(8):2011–2023. https://doi.org/10.1109/TPAMI.2019.2913372

  22. Howard AG, Zhu M, Chen B, et al (2017) MobileNets: Efficient convolutional neural networks for mobile vision Applications. ArXiv170404861 Cs

  23. Redmon J, Farhadi A (2018) YOLOv3: An Incremental improvement. CoRR abs/1804.02767. http://arxiv.org/abs/1804.02767

  24. Lin T-Y, Dollár P, Girshick R et al (2017) Feature Pyramid networks for object detection. IEEE Conf Comput Vis Pattern Recognit CVPR 2017:936–944. https://doi.org/10.1109/CVPR.2017.106

    Article  Google Scholar 

  25. Liu Y, Sun X, Pang JHL (2020) A YOLOv3-based deep learning application research for condition monitoring of rail thermite welded joints. Proc 2020 2nd Int Conf Image Video Signal Process 33–38. https://doi.org/10.1145/3388818.3388827

  26. Liu M, Xie J, Hao J et al (2022) A lightweight and accurate recognition framework for signs of X-ray weld images. Comput Ind 135:103559. https://doi.org/10.1016/j.compind.2021.103559

    Article  Google Scholar 

  27. Zhang X, Wang C, Xiang C, et al (2021) Intelligent Detection method for welding seam defects of automobile wheel hub based on YOLO. Adv Manuf Autom X 693–702. https://doi.org/10.1007/978-981-33-6318-2_86

  28. He T, Zhang Z, Zhang H, Zhang Z, Xie J, Li M (2018) Bag of tricks for image classification with convolutional neural networks. CoRR abs/1812.01187. http://arxiv.org/abs/1812.01187

  29. Liu S, Qi L, Qin H et al (2018) Path Aggregation network for instance segmentation. IEEECVF Conf Comput Vis Pattern Recognit 2018:8759–8768. https://doi.org/10.1109/CVPR.2018.00913

    Article  Google Scholar 

  30. Redmon J, Farhadi A (2017) YOLO9000: better, faster, stronger. IEEE Conf Comput Vis Pattern Recognit CVPR 2017:6517–6525. https://doi.org/10.1109/CVPR.2017.690

    Article  Google Scholar 

  31. Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 4700–4708

  32. Rezatofighi H, Tsoi N, Gwak J et al (2019) Generalized intersection over union: a metric and a loss for bounding box regression. IEEECVF Conf Comput Vis Pattern Recognit CVPR 2019:658–666. https://doi.org/10.1109/CVPR.2019.00075

    Article  Google Scholar 

  33. Ge R, Huang F, Jin C, Yuan Y (2015) Escaping From saddle points --- online stochastic gradient for tensor decomposition. CoRR abs/1503.02101. http://arxiv.org/abs/1503.02101

  34. Dai W, Li D, Tang D et al (2021) Deep learning assisted vision inspection of resistance spot welds. J Manuf Process 62:262–274

    Article  Google Scholar 

  35. Ioffe S, Szegedy C (2015) Batch Normalization: accelerating deep network training by reducing internal covariate shift. ArXiv E-Prints arXiv:1502.03167

  36. Ma N, Zhang X, Zheng HT, Sun J (2018) ShuffleNet V2: Practical guidelines for efficient CNN Architecture design. In: European Conference on Computer Vision

  37. Huang X, Wang X, Lv W, Bai X, Long X, Deng K, Dang Q, Han S, Liu Q, Hu X, Yu D, Ma Y, Yoshie O (2021) PP-YOLOv2: A practical object detector. CoRR abs/2104.10419. https://arxiv.org/abs/2104.10419

  38. Lin T-Y, Maire M, Belongie S, et al (2014) Microsoft COCO: common objects in context. Comput Vis – ECCV 2014 740–755. https://doi.org/10.1007/978-3-319-10602-1_48

  39. Everingham M, Van Gool L, Williams CKI et al (2010) The Pascal visual object classes (VOC) challenge. Int J Comput Vis 88:303–338. https://doi.org/10.1007/s11263-009-0275-4

    Article  Google Scholar 

  40. Yun S, Han D, Oh SJ, et al (2019) CutMix: Regularization strategy to train strong classifiers with localizable features. ArXiv E-Prints arXiv:1905.04899

  41. Misra D (2020) Mish: a self regularized non-monotonic activation function. ArXiv190808681 Cs Stat

  42. Woo S, Park J, Lee J-Y, Kweon IS (2018) Cbam: convolutional block attention module. Proc Eur Conf Comput Vis ECCV 3–19. http://link.springer.com/10.1007/978-3-030-01234-2_1

  43. Zuiderveld K (1994) Contrast limited adaptive histogram equalization. Graph Gems IV 474–485. https://doi.org/10.1016/B978-0-12-336156-1.50061-6

  44. Pong T-C, Shapiro LG, Watson LT, Haralick RM (1984) Experiments in segmentation using a facet model region grower. Comput Vis Graph Image Process 25:1–23. https://doi.org/10.1016/0734-189X(84)90046-X

    Article  Google Scholar 

  45. Loshchilov I, Hutter F (2016) SGDR: Stochastic Gradient Descent With Warm Restarts. CoRR abs/1608.03983. http://arxiv.org/abs/1608.03983

  46. Olague G, Hernández DE, Llamas P et al (2019) Brain programming as a new strategy to create visual routines for object tracking. Multimed Tools Appl 78:5881–5918. https://doi.org/10.1007/s11042-018-6634-9

    Article  Google Scholar 

  47. Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05). IEEE, San Diego, CA, USA, pp 886–893

  48. Platt J (1998) Sequential minimal optimization: a fast algorithm for training support vector machines. Adv Kernel Methods-Support Vector Learn 208. https://www.microsoft.com/en-us/research/publication/sequential-minimal-optimization-a-fast-algorithm-for-training-support-vector-machines/

Download references

Funding

This work was supported by National Natural Science Foundation of China (No. 51675309).

Author information

Authors and Affiliations

  1. MOE Key Lab for Liquid–Solid Structure Evolution and Materials Processing, Institute of Materials Joining, ShanDong University, Jinan, 250061, China

    Sen Li, Jinqiang Gao, Erlong Zhou, Qiong Pan & Xiaofei Wang

Authors
  1. Sen Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jinqiang Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Erlong Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Qiong Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xiaofei Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jinqiang Gao.

Ethics declarations

Ethics approval

Ethical approval was not required for this study.

Consent to participate

Written informed consent was obtained from individual or guardian participants.

Consent to publish

The manuscript was approved by all authors for publication.

Competing interests

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Recommended for publication by Commission XII - Arc Welding Processes and Production Systems

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, S., Gao, J., Zhou, E. et al. Deep learning-based fusion hole state recognition and width extraction for thin plate TIG welding. Weld World 66, 1329–1347 (2022). https://doi.org/10.1007/s40194-022-01287-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40194-022-01287-4

Keywords

Search

Navigation