Skip to main content
SpringerLink
Log in

Optimization of weld penetration prediction based on weld pool image and deep learning approach in gas tungsten arc welding

  • ORIGINAL ARTICLE
  • Published:
The International Journal of Advanced Manufacturing Technology Aims and scope Submit manuscript

Abstract

Gas tungsten arc welding (GTAW) is a popular technology for joining metallic parts with high stability and quality. However, many GTAW-related processes are conducted manually, which is time-consuming, and the weld quality strongly depends on the skill of the welder. Although various automatic GTAW methods and systems have been developed, controlling the weld quality remains a challenge because of the difficulties involved in predicting the weld quality, such as the penetration depth and backside bead geometry, during welding. Hence, this paper proposes an accurate and effective method for estimating the penetration depth through weld pool monitoring using a convolutional neural network (CNN) trained on weld pool images. The weld pool images contained several objects, each influencing the prediction accuracy. The CNN architecture and the structure of the fully connected layers (FCLs) also affected the prediction accuracy. To optimize the performance of the estimation model, the effects of each object in the weld pool image and the structure of the CNN architecture were analyzed and evaluated. The structure of the FCLs that outputted a quantitative penetration depth was optimized and evaluated through hyperparameter tuning. With the proposed method, the optimized model could quantitatively predict the penetration depth; the mean absolute error was 0.0516 mm, with an R2 value of 0.998. Accurately predicting the penetration depth can be employed in real-time weld quality control to ensure a sound weld back bead.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Manorathna RP, Phairatt P, Ogun P, Widjanarko T, Chamberlain M, Justham L, Marimuthu S, Jackson MR (2014) Feature extraction and tracking of a weld joint for adaptive robotic welding. 13th Int Conf Control Autom Robot Vis (ICARCV) 1368–1372. https://doi.org/10.1109/ICARCV.2014.7064515

  2. Kalyankar V, Bhoskar A, Deshmukh D, Patil S (2022) On the performance of metallurgical behaviour of Stellite 6 cladding deposited on SS316L substrate with PTAW process. Can Metall Q 61(2):130–144. https://doi.org/10.1080/00084433.2022.2031681

    Article  Google Scholar 

  3. Bhoskar A, Kalyankar V, Deshmukh D (2023) Metallurgical characterisation of multi-track Stellite 6 coating on SS316L substrate. Can Metall Q 62(4):665–677. https://doi.org/10.1080/00084433.2022.2149009

    Article  Google Scholar 

  4. Kalyankar V, Bhoskar A (2021) Influence of torch oscillation on the microstructure of Colmonoy 6 overlay deposition on SS304 substrate with PTA welding process. Metall Res Technol 118(4):406–418. https://doi.org/10.1051/metal/2021045

    Article  Google Scholar 

  5. Kovacevic R, Zhang YM, Li L (1996) Monitoring of weld joint penetration based on weld pool geometrical appearance. Weld J 75(10):317–329

    Google Scholar 

  6. Bae K, Lee T, Ahn K (2001) An optical sensing system for seam tracking and weld pool control in gas metal arc welding of steel pipe. J Mater Process Technol 120(1):458–465. https://doi.org/10.1016/S0924-0136(01)01216-X

    Article  Google Scholar 

  7. Wang Z, Zhang C, Pan Z, Wang Z, Liu L, Qi X, Mao S, Pan J (2018) Image segmentation approaches for weld pool monitoring during robotic arc welding. Appl Sci 8(12):2445. https://doi.org/10.3390/app8122445

    Article  Google Scholar 

  8. Chen Z, Chen J, Feng Z (2017) Monitoring weld pool surface and penetration using reversed electrode images. Weld J 96:367–375

    Google Scholar 

  9. Kshirsagar R, Jones S, Lawrence J, Tabor J (2019) Prediction of bead geometry using a two-stage SVM–ANN algorithm for automated tungsten inert gas (TIG) welds. J Manuf Mater Process 3(2):39. https://doi.org/10.3390/jmmp3020039

    Article  Google Scholar 

  10. Liu L, Chen H, Chen S (2019) Online monitoring of variable polarity TIG welding penetration state based on fusion of welding characteristic parameters and SVM. Trans Intell Weld Manuf 2(1):87–104. https://doi.org/10.1007/978-981-10-8740-0_5

    Article  Google Scholar 

  11. Ouafi E, Bélanger R, Méthot J (2011) Artificial neural network-based resistance spot welding quality assessment system. Metall Res Technol 108(6):343–355. https://doi.org/10.1051/metal/2011066

    Article  Google Scholar 

  12. Chokkalingham S, Chandrasekhar N, Vasudevan M (2012) Predicting the depth of penetration and weld bead width from the infra red thermal image of the weld pool using artificial neural network modeling. J Intell Manuf 23:1995–2001. https://doi.org/10.1007/s10845-011-0526-4

    Article  Google Scholar 

  13. Mahadevan R, Jagan A, Pavithran L, Shrivastava A, Selvaraj S (2021) Intelligent welding by using machine learning techniques. Mater Today: Proc 46(2):7402–7410. https://doi.org/10.1016/j.matpr.2020.12.1149

    Article  Google Scholar 

  14. Korat P, Sama M (2019) Implementation of artificial intelligence in TIG. Proc Int Conf Advancements Comput Manag (ICACM) 2019:1055–1062. https://doi.org/10.2139/ssrn.3462445

    Article  Google Scholar 

  15. Hong Y, Chang B, Peng G, Yuan Z, Hou X, Xue B, Du D (2018) In-process monitoring of lack of fusion in ultra-thin sheets edge welding using machine vision. Sensors 18(8):2411. https://doi.org/10.3390/s18082411

    Article  Google Scholar 

  16. Fang J, Wang K (2019) Weld pool image segmentation of hump formation based on fuzzy C-means and Chan-Vese model. J Mater Eng Perform 28:4467–4476. https://doi.org/10.1007/s11665-019-04168-y

    Article  Google Scholar 

  17. Jiang C, Zhang F, Wang Z (2017) Image processing of aluminum alloy weld pool for robotic VPPAW based on visual sensing. IEEE Access 5:21567–21573. https://doi.org/10.1109/ACCESS.2017.2761986

    Article  Google Scholar 

  18. Wen H, Zeng J, Bian Z, Hu A, Chu F, Mao Y (2022) Study of weld pool monitoring system based on spatial filtering. J Manuf Process 76:638–645. https://doi.org/10.1016/j.jmapro.2022.02.044

    Article  Google Scholar 

  19. Cheng Y, Wang Q, Jiao W, Yu R, Chen S, Zhang Y, Xiao J (2020) Detecting dynamic development of weld pool using machine learning from innovative composite images for adaptive welding. J Manuf Process 5:908–915. https://doi.org/10.1016/j.jmapro.2020.04.059

    Article  Google Scholar 

  20. Cheng Y, Chen S, Xiao J, Zhang Y (2021) Dynamic estimation of joint penetration by deep learning from weld pool image. Sci Technol Weld Join 26(4):279–285. https://doi.org/10.1080/13621718.2021.1896141

    Article  Google Scholar 

  21. Li C, Wang Q, Jiao W, Johnson M, Zhang Y (2020) Deep learning-based detection of penetration from weld pool reflection images. Weld J 99:239–245. https://doi.org/10.29391/2020.99.022

    Article  Google Scholar 

  22. Nomura K, Fukushima K, Matsumura T, Asai S (2021) Burn-through prediction and weld depth estimation by deep learning model monitoring the molten pool in gas metal arc welding with gap fluctuation. J Manuf Process 61:590–600. https://doi.org/10.1016/j.jmapro.2020.10.019

    Article  Google Scholar 

  23. Shelhamer E, Long J, Darrell T (2017) Fully convolutional networks for semantic segmentation. IEEE Trans Pattern Anal Mach Intell 39:640–651. https://doi.org/10.1109/TPAMI.2016.2572683

    Article  Google Scholar 

  24. Ronneberger O, Fischer P, Brox T (2015) U-Net: convolutional networks for biomedical image segmentation. Medical Image Computing and Computer-Assisted Intervention (MICCAI 2015) 234–241. https://doi.org/10.1007/978-3-319-24574-4_28

  25. Cai W, Jiang P, Shu L, Geng S, Zhou Q (2022) Real-time identification of molten pool and keyhole using a deep learning-based semantic segmentation approach in penetration status monitoring. J Manuf Process 76:695–707. https://doi.org/10.1016/j.jmapro.2022.02.058

    Article  Google Scholar 

  26. Yu R, Kershaw J, Wang P, Zhang Y (2021) Real-time recognition of arc weld pool using image segmentation network. J Manuf Process 72:159–167. https://doi.org/10.1016/j.jmapro.2022.02.058

    Article  Google Scholar 

  27. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. Int Conf Learn Representations (ICLR 2015) https://doi.org/10.48550/arXiv.1409.1556

  28. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Proceedings. 770–778. https://doi.org/10.1109/cvpr.2016.90

  29. Huang G, Liu Z, Maaten L, Weinberger K (2017) Densely connected convolutional networks, 2017 IEEE conference on computer vision and pattern recognition (CVPR). Proceedings: 2261–2269. https://doi.org/10.48550/arXiv.1608.06993

  30. Tan M and Le Q (2019) EfficientNet: rethinking model scaling for convolutional neural networks. 36th International Conference on Machine Learning (ICML 2019). Proceedings: 6105–6114. https://doi.org/10.48550/arXiv.1905.11946

  31. Zhang Y, Cao Z, Kovacevic R (1996) Numerical analysis of fully penetrated weld pools in gas tungsten arc welding. Proc Inst Mech Eng C J Mech Eng Sci 210(2):187–195. https://doi.org/10.1243/PIME_PROC_1996_210_185_02

    Article  Google Scholar 

  32. Baek D, Moon H, Park S (2022) In-process prediction of weld penetration depth using machine learning-based molten pool extraction technique in tungsten arc welding. J Intell Manuf. https://doi.org/10.1007/s10845-022-02013-z

    Article  Google Scholar 

  33. Brochu E, Cora V, Freitas N (2009) A tutorial on Bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning. arXiv: 1012.2599. https://doi.org/10.48550/arXiv.1012.2599

  34. Wu Y, Johnson J (2021) Rethinking “Batch” in BatchNorm. ArXiv: 2105.07576. https://doi.org/10.48550/arXiv.2105.07576

  35. Kingma D, Ba J (2015) Adam: a method for stochastic optimization. 3rd Int Conf Learn Representations (ICLR 2015). https://doi.org/10.48550/arXiv.1412.6980

  36. Chollet F (2015) Keras. https://keras.io

Download references

Funding

This study has been conducted with support from the Korea Institute of Industrial Technology under “The dynamic parameter control based smart welding system module development for the complete joint penetration weld (KITECH EH-23–0007).”

Author information

Authors and Affiliations

  1. Precision Manufacturing and Control R&D Group, Korea Institute of Industrial Technology, Sasang-Gu, Busan, 46938, Republic of Korea

    Daehyun Baek & Hyeong Soon Moon

  2. Graduate School of Mechanical Engineering, Pusan National University, Geumjeong-Gu, Busan, 46241, Republic of Korea

    Daehyun Baek

  3. School of Mechanical Engineering, Pusan National University, Geumjeong-Gu, Busan, 46241, Republic of Korea

    Sang-Hu Park

Authors
  1. Daehyun Baek
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hyeong Soon Moon
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sang-Hu Park
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the study’s conception and design. Material preparation, data collection, and analysis were performed by DB, HSM, and S-HP. The first draft of the manuscript was written by DB, and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Corresponding authors

Correspondence to Hyeong Soon Moon or Sang-Hu Park.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Baek, D., Moon, H.S. & Park, SH. Optimization of weld penetration prediction based on weld pool image and deep learning approach in gas tungsten arc welding. Int J Adv Manuf Technol 130, 2617–2633 (2024). https://doi.org/10.1007/s00170-023-12855-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00170-023-12855-3

Keywords

Search

Navigation