Abstract
Resistance spot welding is the most commonly used welding method in the welding process of automotive body-in-white manufacturing, but the appearance quality of the welding spot still relies on manual inspection, which is inefficient and error-prone. To this end, two methods based on deep learning are proposed to recognize welding spot appearances in this paper. In the first method, a practical convolutional neural network (CNN) model is quickly obtained by fine-tuning the VGG net. In the second method, the Release-Compression (RC) block is designed to fully utilize the power of convolution operation and greatly reduce the parameter number, and the information retention strategies are proposed to optimize the bottom and top of the network, so an ad-hoc CNN model named RswNet is obtained by combining RC block and information retention strategies. Experiment results show that the accuracies of the proposed two models are both higher than existing models, and RswNet has the higher accuracy and its parameters are reduced by more than 56% compared with existing models.
Similar content being viewed by others
References
Ali, H., Rada, L., & Badshah, N. (2018). Image segmentation for intensity inhomogeneity in presence of high noise. IEEE Transactions on Image Processing, 27(8), 3729–3738.
Amiri, N., Farrahi, G. H., Kashyzadeh, K. R., & Chizari, M. (2020). Applications of ultrasonic testing and machine learning methods to predict the static and fatigue behavior of spot-welded joints. Journal of Manufacturing Processes, 52, 26–34.
Bacioiu, D., Melton, G., Papaelias, M., Shaw, R. (2019). Automated defect classification of SS304 TIG welding process using visible spectrum camera and machine learning. NDT and E International 107, 102139.1–102139.9.
Chen, H., Pang, Y., Hu, Q., & Liu, K. (2020). Solar Cell Surface Defect Inspection Based on Multispectral Convolutional Neural Network. Journal of Intelligent Manufacturing, 31(2), 453–468.
Chen, L. C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2018). DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(4), 834–848.
Dai, W., Li, D., Tang, D., Jiang, Q., Wang, D., Wang, H., & Peng, Y. (2021). Deep learning assisted vision inspection of resistance spot welds. Journal of Manufacturing Processes, 62, 262–274.
Espinosa, A. R., Bressan, M., & Giraldo, L. F. (2020). Failure signature classification in solar photovoltaic plants using RGB images and convolutional neural networks - ScienceDirect. Renewable Energy, 162, 249–256.
Fan, J., & Wang, J. (2018). A two-phase fuzzy clustering algorithm based on neurodynamic optimization with its application for PolSAR image segmentation. IEEE Transactions on Fuzzy Systems, 26(1), 72–83.
Gavidel, S. Z., Lu, S., & Rickli, J. L. (2019). Performance analysis and comparison of machine learning algorithms for predicting nugget width of resistance spot welding joints. International Journal of Advanced Manufacturing Technology, 105(9), 3779–3796.
Guo, Z., Ye, S., Wang, Y., Lin, C. (2017). Resistance welding spot defect detection with convolutional neural networks. International Conference on Computer Vision Systems. Springer, Cham, 10528, 169–174.
He, K., Zhang, X., Ren, S., Sun, J. (2016). Deep residual learning for image recognition. In: Proceedings of the 2016 IEEE conference on computer vision and pattern recognition (CVPR), 770–778.
Hu, Y., Soltoggio, A., Lock, R., & Carter, S. (2019). A fully convolutional two-stream fusion network for interactive image segmentation. Neural Networks, 109, 31–42.
Huang, G., Liu, Z., & Maaten, L. V. D. (2017). Densely connected convolutional networks. In: Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2261–2269
Jiang, H., Hu, Q., Zhi, Z., Gao, J., Gao, Z., Wang, R., He, S., & Li, H. Convolution neural network model with improved pooling strategy and feature selection for weld defect recognition. Welding in the World, 2020(1).
Kim, Y., Kim, T., Youn, B. D., Ahn, S. H. (2021). Machining quality monitoring (MQM) in laser-assisted micro-milling of glass using cutting force signals: an image-based deep transfer learning. J Intel Manuf
Kim, M., Lee, M., An, M., & Lee, H. (2020). Effective automatic defect classification process based on CNN with stacking ensemble model for TFT-LCD panel. Journal of Intelligent Manufacturing, 31(5), 1165–1174.
Krizhevsky, A., Sutskever, I., Hinton, G. E. (2012). ImageNet classification with deep convolutional neural networks. In: Proceedings of the 25th International Conference on Neural Information Processing Systems (NIPS), 2012: 1097–1105.
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278–2324.
Li, Y., Cao, G., Yu, Q., & Li, X. (2018). Active contours driven by non-local Gaussian distribution fitting energy for image segmentation. Applied Intelligence, 48(12), 4855–4870.
Lin, M., Chen, Q., Yan, S. (2014). Network in network. In: International Conference on Learning Representations (ICLR).
Lu, L., Shin, Y., Su, Y., & Karniadakis, G. E. (2020). Dying ReLU and Initialization: Theory and Numerical Examples. Communications in Computational Physics, 28(5), 1671–1706.
Martín, Ó., López, M., & Martín, F. (2007). Artificial neural networks for quality control by ultrasonic testing in resistance spot welding. Journal of Materials Processing Tech, 183(2–3), 226–233.
Maskey, A., Cho, M. (2020). CubeSatNet: Ultralight Convolutional Neural Network designed for on-orbit binary image classification on a 1U CubeSat. Engineering Applications of Artificial Intelligence 96.
Miao, J., Huang, T., Zhou, X., Wang, Y., & Liu, J. (2018). Image segmentation based on an active contour model of partial image restoration with local cosine fitting energy. Information Sciences: An International Journal, 447, 52–71.
Molchanov, P., Tyree, S., Karras, T., Aila, T., Kautz, J. (2017). Pruning convolutional neural networks for resource efficient inference. In: International Conference on Learning Representations (ICLR),.
Pereda, M., Santos, J. I., Martín, Ó., & Galán, J. M. (2015). Direct quality prediction in resistance spot welding process Sensitivity, specificity and predictive accuracy comparative analysis. Science and Technology of Welding and Joining, 20(8), 479–685.
Razavian, A. S., Azizpour, H., Sullivan, J., & Carlsson, S. (2014). CNN Features off-the-shelf: An Astounding Baseline for Recognition. IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPR Workshops), 2014, 512–519.
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016). You Only Look Once: Unified, Real-Time Object Detection. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, 779–788.
Shang, J., An, W., Liu, Y., Han, B., & Guo, Y. (2020). Oil Pipeline Weld Defect Identification System Based on Convolutional Neural Network. Ksii Transactions on Internet and Information Systems, 14(3), 1086–1103.
Shelhamer, E., Long, J., & Darrell, T. (2017). Fully Convolutional Networks for Semantic Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(4), 640–651.
Simonyan, K. & Zisserman, A. (2015). Very deep convolutional networks for large-scale image recognition. In: International Conference on Learning Representations (ICLR).
Spampinato, C., Palazzo, S., Giordano, D., Aldinucci, M., & Leonardi, R. (2017). Deep learning for automated skeletal bone age assessment in X-ray images. Medical Image Analysis, 36, 41–51.
Sun, H., Yang, J., & Wang, L. (2016). Resistance spot welding quality identification with particle swarm optimization and a kernel extreme learning machine model. The International Journal of Advanced Manufacturing Technology, 91(5–8), 1879–1887.
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015). Going deeper with convolutions. In: Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 1–9.
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z., (2016) Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2818–2826.
Wang, X., Guan, S., Lin, H., Wang, B., & He, X. (2019). Classification of spot-welded joint strength using ultrasonic signal time-frequency features and PSO-SVM method. Ultrasonics, 91, 161–169.
Xia, Y., Zhou, L., Shen, Y., Wegner, D. M., Haselhuhn, A. S., Li, Y., & Carlson, B. E. (2020). Online measurement of weld penetration in robotic resistance spot welding using electrode displacement signals. Measurement, 168, 108397.
Yan, Y., Liu, D., Gao, B., Tian, G. Y., & Cai, Z. C. (2020). A Deep Learning-Based Ultrasonic Pattern Recognition Method for Inspecting Girth Weld Cracking of Gas Pipeline. IEEE Sensors Journal, 20(14), 7997–8006.
Yang, Y., Pan, L., Ma, J., Yang, R., Zhu, Y., Yang, Y., & Zhang, L. (2020). A High-Performance Deep Learning Algorithm for the Automated Optical Inspection of Laser Welding. Applied Sciences, 10(3), 933.
Ye, S., Guo, Z., Zheng, P., Wang, L., Lin, C. (2017). A vision inspection system for the defects of resistance spot welding based on neural network. In: International Conference on Computer Vision Systems. Springer, Cham 10528, 161–168.
Yu, F., Koltun, V. (2016). Multi-scale context aggregation by dilated convolutions. In: International Conference on Learning Representations (ICLR).
Zaharuddin, M. F. A., Kim, D., & Rhee, S. (2017). An ANFIS based approach for predicting the weld strength of resistance spot welding in artificial intelligence development. Journal of Mechanical Science and Technology, 31(11), 5467–5476.
Zhang, J., Zhang, P. (2012). A SVM regression predicting model for indentation depth of welding spot based on digital image processing. In: Proceedings of SPIE—The International Society for Optical Engineering 8334
Acknowledgements
The presented work was supported by the National Key Research and Development Project of China (no. 2020YFB1713300), the Technological Innovation and Application Development Project of Chongqing (no. cstc2019jscx-mbdxX0056), the National Natural Science Foundation of Chongqing (no. cstc2021jcyj-msxmX0732), and the Fundamental Research Funds for the Central Universities (no. 2021CDJKYJH021).
Author information
Authors and Affiliations
Corresponding authors
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Xiao, M., Yang, B., Wang, S. et al. Research on recognition methods of spot-welding surface appearances based on transfer learning and a lightweight high-precision convolutional neural network. J Intell Manuf 34, 2153–2170 (2023). https://doi.org/10.1007/s10845-022-01909-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10845-022-01909-0