Skip to main content
SpringerLink
Log in

Research on defect detection of toy sets based on an improved U-Net

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

In order to address the problem of low efficiency and accuracy of artificial visual inspection for the quality of toy sets, this paper investigates the defect detection of toy sets based on machine vision. Firstly, an improved U-Net (IU-Net) is developed by introducing the Squeeze-and-Excitation (SE) block into the U-Net and modifying its objective function. Then, a convolutional neural network (CNN) is constructed for feature extraction and defect matching. Finally, a defect detection model for toy sets is created by integrating these two models. In addition, when it comes to a new-variety toy set, the IU-Net model is trained through transfer learning with a small number of new-variety data samples. This can reduce the dependence on new-variety data samples and improve the efficiency of model training process. The experimental results show that the defect detection accuracy of the proposed method is 100% and 99.43% for the whole toy set and single component, respectively, which meets the requirements of industrial automation quality inspection of toy sets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Data availability

The datasets used or analyzed during the current study are available from the corresponding author on reasonable request.

References

  1. Smith, M.L., Smith, L.N., Hansen, M.F.: The quiet revolution in machine vision-a state-of-the-art survey paper, including historical review, perspectives, and future directions. Comput. Ind. 130, 103472 (2021)

    Article  Google Scholar 

  2. Liu, G.H., Zheng, X.T.: Fabric defect detection based on information entropy and frequency domain saliency. Vis. Comput. 37(3), 515–528 (2021). https://doi.org/10.1007/s00371-020-01820-w

    Article  Google Scholar 

  3. Ren, Z., Fang, F., Yan, N., Wu, Y.: State of the art in defect detection based on machine vision. Int. J. Precis. Eng. Manuf.-Green Technol. 9, 1–31 (2021)

    Google Scholar 

  4. Liu, G.H., Li, F.: Fabric defect detection based on low-rank decomposition with structural constraints. Vis. Comput. 38(2), 639–653 (2022). https://doi.org/10.1007/s00371-020-02040-y

    Article  Google Scholar 

  5. Shaik, K.B., Ganesan, P., Kalist, V., Sathish, B.S., Jenitha, J.M.M.: Comparative study of skin color detection and segmentation in HSV and YCbCr Color space. Procedia Comput. Sci. 57, 41–48 (2015). https://doi.org/10.1016/j.procs.2015.07.362

    Article  Google Scholar 

  6. Otsu, N.: A threshold selection method from gray-level histograms. IEEE Trans. Syst. Man Cybern. 9(1), 62–66 (1979)

    Article  Google Scholar 

  7. Tsai, D.-M., Lin, C.-T.: Fast normalized cross correlation for defect detection. Pattern Recogn. Lett. 24(15), 2625–2631 (2003)

    Article  Google Scholar 

  8. Heo, Y.S., Lee, K.M., Lee, S.U.: Robust stereo matching using adaptive normalized cross-correlation. IEEE Trans. Pattern Anal. Mach. Intell. 33(4), 807–822 (2010)

    Google Scholar 

  9. Kamoona, A.M., Gostar, A.K., Bab-Hadiashar, A., Hoseinnezhad, R.: Point pattern feature-based anomaly detection for manufacturing defects, in the random finite set framework. IEEE Access. 9, 158672–158681 (2021). https://doi.org/10.1109/access.2021.3130261

    Article  Google Scholar 

  10. Suvdaa, B., Ahn, J., Ko, J.: Steel surface defects detection and classification using SIFT and voting strategy. Int. J. Softw. Eng. and Appl. 6(2), 161–166 (2012)

    Google Scholar 

  11. Nigam, A., Tripathi, R.C.: Trademark image retrieval using weighted combination of sift and HSV correlogram. Int. J. Comput. Appl. Technol. 54(1), 61–67 (2016)

    Article  Google Scholar 

  12. Huang, Y.B., Qiu, C.Y., Yuan, K.: Surface defect saliency of magnetic tile. Vis. Comput. 36(1), 85–96 (2020). https://doi.org/10.1007/s00371-018-1588-5

    Article  MathSciNet  Google Scholar 

  13. Zheng, X.Q., Zheng, S., Kong, Y.G., Chen, J.: Recent advances in surface defect inspection of industrial products using deep learning techniques. Int. J. Adv. Manuf. Technol. 113(1–2), 35–58 (2021). https://doi.org/10.1007/s00170-021-06592-8

    Article  Google Scholar 

  14. Wang, D., Hu, G.Q., Lyu, C.Z.: FRNet: an end-to-end feature refinement neural network for medical image segmentation. Vis. Comput. 37(5), 1101–1112 (2021). https://doi.org/10.1007/s00371-020-01855-z

    Article  Google Scholar 

  15. Ma, A.L., Wan, Y.T., Zhong, Y.F., Wang, J.J., Zhang, L.P.: SceneNet: Remote sensing scene classification deep learning network using multi-objective neural evolution architecture search. ISPRS-J. Photogramm. Remote Sens. 172, 171–188 (2021). https://doi.org/10.1016/j.isprsjprs.2020.11.025

    Article  Google Scholar 

  16. Chen, W.T., Ouyang, S.B., Yang, J.W., Li, X.J., Zhou, G.D.A., Wang, L.Z.: JAGAN: a framework for complex land cover classification using Gaofen-5 AHSI images. IEEE J. Sel. Top. Appl. Earth Observ. Remote Sens. 15, 1591–1603 (2022). https://doi.org/10.1109/jstars.2022.3144339

    Article  Google Scholar 

  17. Hu, W.H., Wang, T., Wang, Y.S., Chen, Z.Y., Huang, G.H.: LE-MSFE-DDNet: a defect detection network based on low-light enhancement and multi-scale feature extraction. Vis. Comput. 38(11), 3731–3745 (2022). https://doi.org/10.1007/s00371-021-02210-6

    Article  Google Scholar 

  18. Dai, W.T., Erdt, M., Sourin, A.: Detection and segmentation of image anomalies based on unsupervised defect reparation. Vis. Comput. 37(12), 3093–3102 (2021). https://doi.org/10.1007/s00371-021-02257-5

    Article  Google Scholar 

  19. Dai, W.T., Erdt, M., Sourin, A.: Self-supervised pairing image clustering for automated quality control. Vis. Comput. 38(4), 1181–1194 (2022). https://doi.org/10.1007/s00371-021-02137-y

    Article  Google Scholar 

  20. Zhou, G.D.A., Chen, W.T., Gui, Q.S., Li, X.J., Wang, L.Z.: Split depth-wise separable graph-convolution network for road extraction in complex environments from high-resolution remote-sensing images. IEEE Trans. Geosci. Remote Sens. 60, 15 (2022). https://doi.org/10.1109/tgrs.2021.3128033

    Article  Google Scholar 

  21. Wang, Q., Meng, X.Y., Sun, T., Zhang, X.D.: A light iris segmentation network. Vis. Comput. 38(7), 2591–2601 (2022). https://doi.org/10.1007/s00371-021-02134-1

    Article  Google Scholar 

  22. Wang, X.D., Xu, X.W., Wang, Y.L., Wu, P.T., Yan, F., Zeng, Z.Q.: A robust defect detection method for syringe scale without positive samples. Vis. Comput (2022). https://doi.org/10.1007/s00371-022-02671-3

    Article  Google Scholar 

  23. Chen, X., Chen, J., Han, X., Zhao, C., Zhang, D., Zhu, K., et al.: A light-weighted CNN model for wafer structural defect detection. IEEE Access. 8, 24006–24018 (2020). https://doi.org/10.1109/access.2020.2970461

    Article  Google Scholar 

  24. Lin, D., Li, Y., Prasad, S., Nwe, T.L., Dong, S., Oo, Z.M.: CAM-guided multi-path decoding U-net with triplet feature regularization for defect detection and segmentation. Know.-Based Syst. 228, 107272 (2021). https://doi.org/10.1016/j.knosys.2021.107272

    Article  Google Scholar 

  25. Yang, L., Wang, Z., Gao, S.: Pipeline magnetic flux leakage image detection algorithm based on multiscale SSD network. IEEE Trans. Industr. Inf. 16(1), 501–509 (2020). https://doi.org/10.1109/TII.2019.2926283

    Article  Google Scholar 

  26. Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: International Conference on Medical image computing and computer-assisted intervention, pp. 234–241 (2015)

  27. Liu, R.Q., Li, M.H., Shi, J.C., Liang, Y.B.: (2021) fabric defect detection method based on improved U-net. J. Phys.: Conf. Ser. 1, 012160 (1948)

    Google Scholar 

  28. Li, X., Chen, H., Qi, X., Dou, Q., Fu, C.-W., Heng, P.-A.: H-DenseUNet: hybrid densely connected UNet for liver and tumor segmentation from CT volumes. IEEE Trans. Med. Imaging 37(12), 2663–2674 (2018). https://doi.org/10.1109/tmi.2018.2845918

    Article  Google Scholar 

  29. Li, W., Qin, S., Li, F., Wang, L.: MAD-UNet: a deep U-shaped network combined with an attention mechanism for pancreas segmentation in CT images. Med. Phys. 48(1), 329–341 (2021)

    Article  Google Scholar 

  30. Huang, H., Lin, L., Tong, R., Hu, H., Zhang, Q., Iwamoto, Y., et al.: Unet 3+: a full-scale connected unet for medical image segmentation. In: ICASSP 2020–2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1055–1059 (2020)

  31. Qin, X., Zhang, Z., Huang, C., Dehghan, M., Zaiane, O.R., Jagersand, M.: U2-Net: going deeper with nested U-structure for salient object detection. Pattern Recognit. 106, 107404 (2020). https://doi.org/10.1016/j.patcog.2020.107404

    Article  Google Scholar 

  32. Milletari, F., Navab, N., Ahmadi, S.-A.: V-net: Fully convolutional neural networks for volumetric medical image segmentation. In: 2016 fourth international conference on 3D vision (3DV), pp. 565–571 (2016)

  33. Chen, Z.Y., Huang, G.H., Wang, Y., Qiu, J.H., Yang, F., Yu, Z.W., et al.: Bi-deformation-UNet: recombination of differential channels for printed surface defect detection. Vis. Comput (2022). https://doi.org/10.1007/s00371-077-07554-7

    Article  Google Scholar 

  34. Wang, X.J., Hua, Z., Li, J.J.: Cross-UNet: dual-branch infrared and visible image fusion framework based on cross-convolution and attention mechanism. Vis. Comput (2022). https://doi.org/10.1007/s00371-022-02628-6

    Article  Google Scholar 

  35. Cheng, Z.M., Qu, A.P., He, X.F.: Contour-aware semantic segmentation network with spatial attention mechanism for medical image. Vis. Comput. 38(3), 749–762 (2022). https://doi.org/10.1007/s00371-021-02075-9

    Article  Google Scholar 

  36. Kiran, I., Raza, B., Ijaz, A., Khan, M.A.: DenseRes-Unet: segmentation of overlapped/clustered nuclei from multi organ histopathology images. Comput. Biol. Med. 143, 105267 (2022)

    Article  Google Scholar 

  37. Zhang, Z., Wu, C., Coleman, S., Kerr, D.: DENSE-INception U-net for medical image segmentation. Comput. Methods Programs Biomed. 192, 105395 (2020)

    Article  Google Scholar 

  38. Wang, Z., Zou, Y., Liu, P.X.: Hybrid dilation and attention residual U-Net for medical image segmentation. Comput. Biol. Med. 134, 104449 (2021)

    Article  Google Scholar 

  39. Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 7132–7141 (2018)

  40. Kamal, U., Tonmoy, T.I., Das, S., Hasan, M.K.: Automatic traffic sign detection and recognition using SegU-net and a modified Tversky loss function with L1-constraint. IEEE Trans. Intell. Transp. Syst. 99, 1–13 (2019)

    Google Scholar 

  41. Milletari, F., Navab, N., Ahmadi, S.: V-Net: fully convolutional neural networks for volumetric medical image segmentation. In: 2016 Fourth International Conference on 3D Vision (3DV), pp. 565–571 (2016). doi:https://doi.org/10.1109/3DV.2016.79

  42. Shin, H.C., Roth, H.R., Gao, M., Lu, L., Xu, Z., Nogues, I., et al.: Deep convolutional neural networks for computer-aided detection: cnn architectures, dataset characteristics and transfer learning. IEEE Trans. Med. Imaging 35(5), 1285–1298 (2016). https://doi.org/10.1109/TMI.2016.2528162

    Article  Google Scholar 

  43. Baktashmotlagh, M., Harandi, M.T., Lovell, B.C., Salzmann, M.: Unsupervised domain adaptation by domain invariant projection. In: Proceedings of the 2013 IEEE International Conference on Computer Vision, pp. 769–776 (2013). doi:https://doi.org/10.1109/iccv.2013.100.

  44. Yosinski, J., Clune, J., Bengio, Y., Lipson, H.: How transferable are features in deep neural networks? arXiv e-prints. arXiv:1411.1792 (2014)

  45. Oquab, M., Bottou, L., Laptev, I., Sivic, J.: Learning and transferring mid-level image representations using convolutional neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1717–1724 (2014)

  46. Tanner, M.A., Wong, W.H.: The calculation of posterior distributions by data augmentation. J. Am. Stat. Assoc. 82(398), 528–540 (1987)

    Article  MathSciNet  Google Scholar 

  47. Dahl, G.E., Sainath, T.N., Hinton, G.E.: Improving deep neural networks for LVCSR using rectified linear units and dropout. In: 2013 IEEE international conference on acoustics, speech and signal processing, pp. 8609–8613 (2013)

Download references

Funding

This work is supported by the China-Japan Science and Technology Joint Committee of the Ministry of Science and Technology of the People’s Republic of China (Grant No. 2017YFE0128400), the National Natural Science Foundation of China (Grant No. 51905162), the Foundation for Innovative Research Groups of the National Natural Science Foundation of China (Grant No. 51621004), the Project of Science and Technology of Changsha (Grant No. kq2001015), the Key Research and Development program of Hunan Province (2021GK2007), and the program of Leading Scientific and Technological Innovation in High-tech Industries (2021GK4028).

Author information

Authors and Affiliations

  1. College of Mechanical and Vehicle Engineering, Hunan University, Changsha, 410082, China

    Dezhi Yang, Ning Chen, Qiqi Tang, Hang Zhang & Jian Liu

Authors
  1. Dezhi Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ning Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qiqi Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jian Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed in writing and revising the manuscripts.

Corresponding author

Correspondence to Ning Chen.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, D., Chen, N., Tang, Q. et al. Research on defect detection of toy sets based on an improved U-Net. Vis Comput 40, 1095–1109 (2024). https://doi.org/10.1007/s00371-023-02834-w

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-023-02834-w

Keywords

Search

Navigation