Skip to main content
SpringerLink
Log in

Inspection of Welding Defect Based on Multi-feature Fusion and a Convolutional Network

  • Published:
Journal of Nondestructive Evaluation Aims and scope Submit manuscript

Abstract

Robot welding is a basic but indispensable technology for many industries in modern manufacturing. However, many welding parameters affect welding quality. During the real welding process, welding defects are inevitably generated that affect the structural strengths and comprehensive performances of different welding products. Therefore, an accurate welding defect recognition algorithm is necessary for automatic robot welding to assess the effects of defects on structural properties and system maintenance. Much work has been devoted to welding defect recognition. It can be mainly divided into two categories: feature-based and deep learning-based methods. The detection performances of feature-based methods rely on effective image features and strong classifiers. However, faced with weak-textured and weak-contrast welding images, the realization of strong image feature expression still faces a certain challenge. Deep learning-based methods can provide end-to-end detection schemes for welding robots. Nevertheless, an effective deep network model relies on much training data that are not easily collected during real manufacturing. To address the above issues regarding defect detection, a novel welding defect recognition algorithm is proposed based on multi-feature fusion for accurate defect detection based on X-ray images. To improve network training, an effective data augmentation process is proposed to construct the dataset. Combined with transfer learning, the multi-scale features of welding images are acquired for effective feature expression with the pre-trained AlexNet network. On this basis, based on multi-feature fusion, a welding defect recognition algorithm fused to a support vector machine with Dempster–Shafer evidence theory is proposed for multi-scale defect detection. Experiments show that the proposed method achieves a better recognition performance in terms of detecting welding defects than those of other related recognition algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Stavridis, J., Papacharalampopoulos, A., Stavropoulos, P.: Quality assessment in laser welding: a critical review. Int. J. Adv. Manuf. Technol. 94(5–8), 1825–1847 (2018)

    Article  Google Scholar 

  2. Tao, X., Wang, Z., Zhang, Z., Zhang, D., Xu, D., Gong, X., Zhang, L.: Wire defect recognition of spring-wire socket using multitask convolutional neural networks. IEEE Trans. Compon. Packag. Manuf. Technol. 8(4), 689–698 (2018)

    Article  Google Scholar 

  3. Alfaro, S.C., Franco, F.D.: Exploring infrared sensoring for real time welding defects monitoring in GTAW. Sensors 10(6), 5962–5974 (2010)

    Article  Google Scholar 

  4. Jain, D.K., et al.: An evaluation of deep learning based object detection strategies for threat object detection in baggage security imagery. Patt. Recogn. Lett. 120, 112–119 (2019)

    Article  Google Scholar 

  5. Duan, F., Yin, S., Song, P., Zhang, W., Zhu, C., Yokoi, H.: Automatic welding defect detection of X-ray images by using cascade adaboost with penalty term. IEEE Access 7, 125 929-125 938 (2019)

    Article  Google Scholar 

  6. Roy, R.B., Ghosh, A., Bhattacharyya, S., Mahto, R.P., Kumari, K., Pal, S.K., Pal, S.: Weld defect identification in friction stir welding through optimized wavelet transformation of signals and validation through X-ray micro-CT scan. Int. J. Adv. Manuf. Technol. 99(1–4), 623–633 (2018)

    Article  Google Scholar 

  7. Zhan, X., Zhang, D., Yu, H., Chen, J., Li, H., Wei, Y.: Research on X-ray image processing technology for laser welded joints of aluminum alloy. Int. J. Adv. Manuf. Technol. 99(1–4), 683–694 (2018)

    Article  Google Scholar 

  8. Fan, J., Jing, F., Fang, Z., Tan, M.: Automatic recognition system of welding seam type based on SVM method. Int. J. Adv. Manuf. Technol. 92(1–4), 989–999 (2017)

    Article  Google Scholar 

  9. Fioravanti, C.C.B., Centeno, T.M., Da Silva, M.R.D.B. et al.: A deep artificial immune system to detect weld defects in DWDI radiographic images of petroleum pipes. IEEE Access 7, 180 947–180 964 (2019)

  10. Liu, Y.-K., Zhang, Y.-M.: Supervised learning of human welder behaviors for intelligent robotic welding. IEEE Trans. Autom. Sci. Eng. 14(3), 1532–1541 (2015)

    Article  Google Scholar 

  11. Li, Y., Li, Y.F., Wang, Q.L., Xu, D., Tan, M.: Measurement and defect detection of the weld bead based on online vision inspection. IEEE Trans. Instrum. Meas. 59(7), 1841–1849 (2009)

    Google Scholar 

  12. Zhang, L., Ye, Q., Yang, W., Jiao, J.: Weld line detection and tracking via spatial-temporal cascaded hidden Markov models and cross structured light. IEEE Trans. Instrum. Meas. 63(4), 742–753 (2013)

    Article  Google Scholar 

  13. Jia, N., Li, Z., Ren, J., Wang, Y., Yang, L.: A 3d reconstruction method based on grid laser and gray scale photo for visual inspection of welds. Optics Laser Technol. 119, 105648 (2019)

    Article  Google Scholar 

  14. Quanying, D., Shanben, C., Tao, L.: Inspection of weld shape based on the shape from shading. Int. J. Adv. Manuf. Technol. 27(7–8), 667–671 (2006)

    Article  Google Scholar 

  15. Chen, B., Fang, Z., Xia, Y., Zhang, L., Huang, Y., Wang, L.: Accurate defect detection via sparsity reconstruction for weld radiographs. NDT & E Int. 94, 62–69 (2018)

    Article  Google Scholar 

  16. Du, D., Cai, G.-r., Tian, Y., Hou, R.-s., Wang, L.: Automatic inspection of weld defects with X-ray real-time imaging. In: Robotic Welding, Intelligence and Automation. Springer, pp. 359–366 (2007)

  17. Cui, K., Jing, X.: Research on prediction model of geotechnical parameters based on bp neural network. Neural Comput. Appl. 31(12), 8205–8215 (2019)

    Article  Google Scholar 

  18. Yang, L., Li, E., Long, T., Fan, J., Mao, Y., Fang, Z., Liang, Z.: A welding quality detection method for arc welding robot based on 3d reconstruction with SFS algorithm. Int. J. Adv. Manuf. Technol. 94(1–4), 1209–1220 (2018)

    Article  Google Scholar 

  19. Kasban, H., Zahran, O., Arafa, H. , El-Kordy, M., Elaraby, S. M., Abd El-Samie, F.: Welding defect detection from radiography images with a cepstral approach. Ndt & E Int. 44(2), 226–231 (2011)

  20. Das, B., Pal, S., Bag, S.: Weld quality prediction in friction stir welding using wavelet analysis. Int. J. Adv. Manuf. Technol. 89(1–4), 711–725 (2017)

    Article  Google Scholar 

  21. Chen, F.-C., Jahanshahi, M.R.: Nb-cnn: deep learning-based crack detection using convolutional neural network and Naïve Bayes data fusion. IEEE Trans. Ind. Electron. 65(5), 4392–4400 (2017)

    Article  Google Scholar 

  22. Ren, R., Hung, T., Tan, K.C.: A generic deep-learning-based approach for automated surface inspection. IEEE Trans. Cybern. 48(3), 929–940 (2017)

    Article  Google Scholar 

  23. Yang, Y., Yang, R., Pan, L., Ma, J., Zhu, Y., Diao, T., Zhang, L.: A lightweight deep learning algorithm for inspection of laser welding defects on safety vent of power battery. Comput. Ind. 123, 103306 (2020)

    Article  Google Scholar 

  24. Zhang, Z., Wen, G., Chen, S.: Weld image deep learning-based on-line defects detection using convolutional neural networks for al alloy in robotic arc welding. J. Manuf. Process. 45, 208–216 (2019)

    Article  Google Scholar 

  25. Sassi, P., Tripicchio, P., Avizzano, C.A.: A smart monitoring system for automatic welding defect detection. IEEE Trans. Ind. Electron. 66(12), 9641–9650 (2019)

    Article  Google Scholar 

  26. Yang, L., Liu, Y., Peng, J.: An automatic detection and identification method of welded joints based on deep neural network. IEEE Access 7, 164 952–164 961 (2019)

  27. Günther, J., Pilarski, P.M., Helfrich, G., Shen, H., Diepold, K.: Intelligent laser welding through representation, prediction, and control learning: an architecture with deep neural networks and reinforcement learning. Mechatronics 34, 1–11 (2016)

    Article  Google Scholar 

  28. Yang, J., Fu, G., Zhu, W., Cao, Y., Cao, Y., Yang, M.Y.: A deep learning-based surface defect inspection system using multi-scale and channel-compressed features. IEEE Trans. Instrum. Meas. (2020)

  29. Gao, Y., Gao, L., Li, X., Wang, X.V.: A multi-level information fusion-based deep leaning method for vision-based defect recognition. IEEE Trans. Instrum. Meas. (2019)

  30. Makantasis, K., Protopapadakis, E., Doulamis, A., Doulamis, N., Loupos, C.: Deep convolutional neural networks for efficient vision based tunnel inspection. In: Proceedings of IEEE International Conference on Intelligent Computer Communication and Processing (ICCP). IEEE, pp. 335–342 (2015)

  31. Zou, Q., Zhang, Z., Li, Q., Qi, X., Wang, Q., Wang, S.: Deepcrack: learning hierarchical convolutional features for crack detection. IEEE Trans. Image Process. 28(3), 1498–1512 (2018)

    Article  MathSciNet  Google Scholar 

  32. Gong, Y., Shao, H., Luo, J., Li, Z.: A deep transfer learning model for inclusion defect detection of aeronautics composite materials. Compos. Struct. 252, 112681 (2020)

    Article  Google Scholar 

  33. Mery, D., Riffo, V., Zscherpel, U., Mondragón, G., Lillo, I., Zuccar, I., Lobel, H., Carrasco, M.: Gdxray: the database of X-ray images for nondestructive testing. J. Nondestruct. Eval. 34(4), 42 (2015)

    Article  Google Scholar 

  34. Liu, B., Zhang, X., Gao, Z., Chen, L.: Weld defect images classification with vgg16-based neural network. In: Proceedings of International Forum on Digital TV and Wireless Multimedia Communications. Springer, pp. 215–223 (2017)

  35. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), vol. 1. IEEE, pp. 886–893 (2005)

  36. Ojala, T., Pietikainen, M., Maenpaa, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. Patt. Anal. Mach. Intell. 24(7), 971–987 (2002)

    Article  Google Scholar 

  37. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp. 1097–1105 (2012)

  38. Zapata, J., Vilar, R., Ruiz, R.: Automatic inspection system of welding radiographic images based on ANN under a regularisation process. J. Nondestruct. Eval. 31(1), 34–45 (2012)

    Article  Google Scholar 

  39. Liu, J., Fan, Z., Olsen, S.I., Christensen, K.H., Kristensen, J.K.: Boosting active contours for weld pool visual tracking in automatic arc welding. IEEE Trans. Autom. Sci. Eng. 14(2), 1096–1108 (2015)

    Article  Google Scholar 

  40. Wang, G., Liao, T.W.: Automatic identification of different types of welding defects in radiographic images. Ndt & E Int. 35(8), 519–528 (2002)

    Article  Google Scholar 

  41. You, D., Gao, X., Katayama, S.: Wpd-pca-based laser welding process monitoring and defects diagnosis by using fnn and svm. IEEE Trans. Ind. Electron. 62(1), 628–636 (2014)

    Article  Google Scholar 

  42. Kuang, H., Chen, L., Gu, F., Chen, J., Chan, L., Yan, H.: Combining region-of-interest extraction and image enhancement for nighttime vehicle detection. IEEE Intell. Syst. 31(3), 57–65 (2016)

    Article  Google Scholar 

  43. Simonyan,K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprintarXiv:1409.1556 (2014)

  44. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–9 (2015)

  45. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.-C.: Mobilenetv2: inverted residuals and linear bottlenecks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4510–4520 (2018)

  46. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778 (2016)

  47. Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2818–2826 (2016)

Download references

Acknowledgements

The authors wish to thank the anonymous reviewers for their valuable comments and suggestions.

Author information

Authors and Affiliations

  1. School of Electrical Engineering, Zhengzhou University and the Robot Perception and Control Engineering Laboratory of Henan Province, Zhengzhou, 450001, China

    Lei Yang, Benyan Huo & Yanhong Liu

  2. The State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, 100190, China

    Junfeng Fan

Authors
  1. Lei Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Junfeng Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Benyan Huo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yanhong Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yanhong Liu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work was supported by the National Natural Science Foundation of China (No. 62003309), the National Key Research & Development Project of China (2020YFB1313701), Science & Technology Research Project in Henan Province of China (No. 202102210098) and Outstanding Foreign Scientist Support Project in Henan Province of China (No. GZS2019008)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, L., Fan, J., Huo, B. et al. Inspection of Welding Defect Based on Multi-feature Fusion and a Convolutional Network. J Nondestruct Eval 40, 90 (2021). https://doi.org/10.1007/s10921-021-00823-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10921-021-00823-4

Keywords

Search

Navigation