Skip to main content
SpringerLink
Log in

Penetration/keyhole status prediction and model visualization based on deep learning algorithm in plasma arc welding

  • ORIGINAL ARTICLE
  • Published:
The International Journal of Advanced Manufacturing Technology Aims and scope Submit manuscript

Abstract

Accurate keyhole status prediction is critical for realizing the closed-loop control of the keyhole plasma arc welding (K-PAW) processes for acquiring full-penetration weld joints with high efficiency. Visually captured weld pool images from topside provide sufficient information of the liquid metal as well as keyhole behaviors. Weld pool, plasma arc, and keyhole entrance could be clearly recognized reflecting the different features during different keyholing stages. It was proposed to extract the image features automatically based on a deep learning algorithm rather than manually selecting characteristic parameters. Since directly training the deep CNN (convolutional neural network) model using the acquired data led to convergence failure, a well-trained generalized model was employed and fine-tuned accordingly to more easily extract the K-PAW image features. Model training was conducted using obtained dataset, which took weld pool images as input and penetration/keyhole status (partial penetration with a blind keyhole or full penetration with a through keyhole) as output. Underlying correlations between the penetration/keyhole status and topside weld pool images were established. For further verifying the effectiveness and reliability of the trained model, experiments were designed acquiring typical slow keyholing under constant welding current and rapid keyhole switching under pulse welding current. Based on the given data, the verified 90% accuracy was achieved for correctly predicting the keyhole/penetration status. Finally, the visualization of the convolutional layers was carried out, and displayed the features clearly, which is of great significance for understanding the internal mechanism of the neural network.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20

Similar content being viewed by others

Data availability

The datasets used or analyzed during the current study are available from the corresponding author on reasonable request.

References

  1. Metcalfe JC, Quigley MBC (1975) Heat-transfer in plasma-arc welding. Weld J 54:S99–S103

    Google Scholar 

  2. Zhang YM, Zhang SB (1999) Observation of the keyhole during plasma arc welding. Weld J 78:53s–58s

    Google Scholar 

  3. Wu CS, Jia CB, Chen MA (2010) A control system for keyhole plasma arc welding of stainless steel plates with medium thickness. Weld J 89:225s–231s

    Google Scholar 

  4. Martikainen JK, Moisio TJI (1993) Investigation of the effect of welding parameters on weld quality of plasma-arc keyhole welding of structural-steels. Weld J 72:S329–S340

    Google Scholar 

  5. Zhang YM, Liu YC (2007) Control of dynamic keyhole welding process. Automatica 43:876–884. https://doi.org/10.1016/j.automatica.2006.11.008

    Article  MathSciNet  MATH  Google Scholar 

  6. Steffens HD, Kayser IH (1972) Automatic-control for plasma arc-welding with constant keyhole diameter. Weld J 51:408

    Google Scholar 

  7. Zhang SB, Zhang YM (2001) Efflux plasma charge-based sensing and control of joint penetration during keyhole plasma arc welding. Weld J 80:157s–162s

    Google Scholar 

  8. Saad E, Wang HJ, Kovacevic R (2006) Classification of molten pool modes in variable polarity plasma arc welding based on acoustic signature. J Mater Process Technol 174:127–136. https://doi.org/10.1016/j.jmatprotec.2005.03.020

    Article  Google Scholar 

  9. Zhang YM, Zhang SB, Liu YC (2001) A plasma cloud charge sensor for pulse keyhole process control. Meas Sci Technol 12:1365–1370. https://doi.org/10.1088/0957-0233/12/8/352

    Article  Google Scholar 

  10. Zhang GK, Chen J, Wu CS (2017) Simultaneous sensing of weld pool and keyhole in controlled-pulse PAW: the behaviors of the keyhole and the weld pool in plasma arc welding can be used to indicate weld quality. Weld J 96:95s–103s

    Google Scholar 

  11. Zhang WJ, Xiao J, Zhang YM (2016) A mobile sensing system for real-time 3D weld pool surface measurement in manual GTAW. Meas Sci Technol 27:045102. https://doi.org/10.1088/0957-0233/27/4/045102

    Article  Google Scholar 

  12. Saeed G, Zhang YM (2007) Weld pool surface depth measurement using a calibrated camera and structured light. Meas Sci Technol 18:2570–2578. https://doi.org/10.1088/0957-0233/18/8/033

    Article  Google Scholar 

  13. Wu CS, Zhong LM, Gao JQ (2009) Visualization of hump formation in high-speed gas metal arc welding. Measurement Sci Technol 20. https://doi.org/10.1088/0957-0233/20/11/115702

  14. Fabbro R, Slimani S, Coste F, Briand F (2005) Study of keyhole behaviour for full penetration Nd-Yag CW laser welding. J Physics D-Appl Physics 38:1881–1887. https://doi.org/10.1088/0022-3727/38/12/005

    Article  Google Scholar 

  15. Krasnoperov MY, Pieters RRGM, Richardson IM (2004) Weld pool geometry during keyhole laser welding of thin steel sheets. Sci Technol Weld Join 9:501–506. https://doi.org/10.1179/136217104225021733

    Article  Google Scholar 

  16. Kovacevic R, Zhang YM, Li L (1996) Monitoring of weld joint penetration based on weld pool geometrical appearance. Weld J 75:S317–S329

    Google Scholar 

  17. Zhang G, Shi Y, Gu YF, Fan D (2017) Welding torch attitude-based study of human welder interactive behavior with weld pool in GTAW. Robot Comput Integr Manuf 48:145–156. https://doi.org/10.1016/j.rcim.2017.03.009

    Article  Google Scholar 

  18. Fan CJ, Lv FL, Chen SB (2009) Visual sensing and penetration control in aluminum alloy pulsed GTA welding. Int J Adv Manuf Technol 42:126–137. https://doi.org/10.1007/s00170-008-1587-1

    Article  Google Scholar 

  19. Zhang W, Liu Y, Wang X, Zhang YM (2012) Characterization of three-dimensional weld pool surface in GTAW. Weld J 91:195s–203s

    Google Scholar 

  20. Wu S, Gao H, Zhang W, Zhang YM (2017) Analytic weld pool model calibrated by measurements - part 1: principles. Weld J 96:193s–202s

    Google Scholar 

  21. Li C, Wang Q, Jiao W, Johnson M, Zhang YM (2020) Deep learning-based detection of penetration from weld pool reflection images. Weld J 99:239s–245s. https://doi.org/10.29391/2020.99.022

    Article  Google Scholar 

  22. Shevchik S, Le-Quang T, Meylan B, Farahani FV, Olbinado MP, Rack A, Masinelli G, Leinenbach C, Wasmer K (2020) Supervised deep learning for real-time quality monitoring of laser welding with X-ray radiographic guidance. Sci Rep 10. https://doi.org/10.1038/s41598-020-60294-x

  23. Zeiler MD, Taylor GW, Fergus R (2011) Adaptive deconvolutional networks for mid and high level feature learning. 2011 Ieee International Conference on Computer Vision (Iccv): 2018–2025

  24. Mahendran A, Vedaldi A (2015) Understanding deep image representations by inverting them. 2015 Ieee Conference on Computer Vision and Pattern Recognition (Cvpr): 5188–5196. https://doi.org/10.1109/cvpr.2015.7299155

  25. Jia CB, Liu XF, Wu CS, Lin SB (2018) Stereo analysis on the keyhole and weld pool behaviors in K-PAW with triple CCD cameras. J Manuf Process 32:754–762. https://doi.org/10.1016/j.jmapro.2018.03.026

    Article  Google Scholar 

  26. Liu XF, Jia CB, Wu CS, Zhang GK, Gao JQ (2017) Measurement of the keyhole entrance and topside weld pool geometries in keyhole plasma arc welding with dual CCD cameras. J Mater Process Technol 248:39–48. https://doi.org/10.1016/j.jmatprotec.2017.05.012

    Article  Google Scholar 

  27. Hinton GE, Srivastava N, Krizhevsky A, Sutskever I, Salakhutdinov RR (2012) Improving neural networks by preventing co-adaptation of feature detectors. Comput Therm Sci 4:212–223

    Google Scholar 

  28. Krizhevsky A, Sutskever I, Hinton GE (2017) ImageNet classification with deep convolutional neural networks. Commun ACM 60:84–90. https://doi.org/10.1145/3065386

    Article  Google Scholar 

  29. Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15:1929–1958

    MathSciNet  MATH  Google Scholar 

  30. Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. 2014 Ieee Conference on Computer Vision and Pattern Recognition (Cvpr): 580–587. https://doi.org/10.1109/Cvpr.2014.81

  31. Zeiler MD, Fergus R (2014) Visualizing and understanding convolutional networks. Comp Vis - Eccv 2014, Pt I 8689:818–833. https://doi.org/10.1007/978-3-319-10590-1-53

    Article  Google Scholar 

  32. Yosinski J, Clune J, Nguyen AM, Fuchs TJ, Lipson H (2015) Understanding neural networks through deep visualization. Comp Vis Pattern Recognit. arXiv:1506.06579 [cs.CV]

Download references

Funding

This work was supported by the National Natural Science Foundation of China (grant no. 51975332), the Major Scientific and Technological Innovation Project of Shandong Province (2019JZZY010358), and the Aeronautical Science Foundation of China (201742Q3001).

Author information

Authors and Affiliations

  1. MOE Key Lab for Liquid-Solid Structure Evolution and Materials Processing, Institute of Materials Joining, Shandong University, Jinan, 250061, China

    Chuan-Bao Jia, Yong Zhang, Chang-Hai Yu & Chuan-Song Wu

  2. School of Computer Science and Technology, Shandong Jianzhu University, Jinan, 250101, China

    Xin-Feng Liu

  3. Leibniz Institute for Plasma Science and Technology, 17489, Greifswald, Germany

    Guo-Kai Zhang

Authors
  1. Chuan-Bao Jia
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xin-Feng Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guo-Kai Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Chang-Hai Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Chuan-Song Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Chuanbao Jia: funding acquisition, project administration, resources, and supervision, and writing original draft. Xinfeng Liu: algorithm design, welding experiment. Guokai Zhang: welding experiment, and data curation. Yong Zhang: supplement experiments. Changhai Yu: complete and improve manuscript. Chuansong Wu: writing review and editing.

Corresponding author

Correspondence to Chuan-Bao Jia.

Ethics declarations

Ethical approval

Ethical approval was not required for this study.

Consent to participate

Written informed consent was obtained from individual or guardian participants.

Consent to publish

The manuscript was approved by all authors for publication.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

Table 6 The penetration forecast result of test case 1
Full size table
Table 7 The penetration forecast result of test case 2
Full size table
Table 8 The penetration forecast result under square-wave current condition
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jia, CB., Liu, XF., Zhang, GK. et al. Penetration/keyhole status prediction and model visualization based on deep learning algorithm in plasma arc welding. Int J Adv Manuf Technol 117, 3577–3597 (2021). https://doi.org/10.1007/s00170-021-07903-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00170-021-07903-9

Keywords

Search

Navigation