Skip to main content
SpringerLink
Log in

Edge devices-oriented surface defect segmentation by GhostNet Fusion Block and Global Auxiliary Layer

  • Research
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

This paper introduces an approach for the segmentation of surface defects, referred to as Efficient Surface Defect Network (ESD-Net). The proposed method uses novel components called the GhostNet Fusion Block (GFB) and the Global Auxiliary Layer (GAL) to make it edge computing-ready and to increase its performance on segmentation. The GFB algorithm employs a technique whereby it conserves and combines feature maps of reduced resolution from the original image with feature maps that have been downsampled at various resolutions. Moreover, the GAL amplifies the GFB by including comprehensive contextual information from a global perspective. The experiment shows that the proposed method outperforms state-of-the-art algorithms on SD-saliency-900, MSD, and Magnetic-tile, three public surface defect datasets with mIoU of \(82.4\%\), \(92.9\%\), and \(78.8\%\), respectively. Embedded device experiments have proven that ESDNet can be utilized on a wider range of cost-effective industrial devices with acceptable latency.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Zheng, Z., Hu, Y., Zhang, Y., Yang, H., Qiao, Y., Qu, Z., Huang, Y.: Casppnet: a chained atrous spatial pyramid pooling network for steel defect detection. Meas. Sci. Technol. 33(8), 085403 (2022). https://doi.org/10.1088/1361-6501/ac68d2

    Article  ADS  Google Scholar 

  2. Tang, B., Chen, L., Sun, W., Lin, Z.K.: Review of surface defect detection of steel products based on machine vision. IET Image Process. 17(2), 303–322 (2023). https://doi.org/10.1049/ipr2.12647

    Article  Google Scholar 

  3. Liu, G.: Surface defect detection methods based on deep learning: a brief review. In: 2020 2nd International Conference on Information Technology and Computer Application (ITCA), (2020). pp. 200–203 (2020). https://doi.org/10.1109/ITCA52113.2020.00049

  4. Poxi, H., Chen, W., Gao, J.: Overview of surface defect detection methods based on deep learning. In: Wang, Y., Yu, T., Wang, K. (eds) Advanced Manufacturing and Automation XII, (2023). pp. 123–128 (2023)

  5. Minaee, S., Boykov, Y., Porikli, F., Plaza, A., Kehtarnavaz, N., Terzopoulos, D.: Image segmentation using deep learning: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 44(7), 3523–3542 (2022). https://doi.org/10.1109/TPAMI.2021.3059968

    Article  PubMed  Google Scholar 

  6. Bao, Y., Song, K., Liu, J., Wang, Y., Yan, Y., Yu, H., Li, X.: Triplet-graph reasoning network for few-shot metal generic surface defect segmentation. IEEE Trans. Instrum. Meas. 70, 1–11 (2021). https://doi.org/10.1109/TIM.2021.3083561

    Article  Google Scholar 

  7. Lu, P., Jing, J., Huang, Y.: Mrd-net: an effective cnn-based segmentation network for surface defect detection. IEEE Trans. Instrum. Meas. 71, 1–12 (2022). https://doi.org/10.1109/TIM.2022.3200361

    Article  Google Scholar 

  8. Liu, T., He, Z., Lin, Z., Cao, G.Z., Su, W., Xie, S.: An adaptive image segmentation network for surface defect detection. IEEE Trans. Neural Netw. Learn. Syst. (2022). https://doi.org/10.1109/TNNLS.2022.3230426

    Article  PubMed  PubMed Central  Google Scholar 

  9. Cao, J., Yang, G., Yang, X.: A pixel-level segmentation convolutional neural network based on deep feature fusion for surface defect detection. IEEE Trans. Instrum. Meas. 70, 1–12 (2021). https://doi.org/10.1109/TIM.2020.3033726

    Article  CAS  Google Scholar 

  10. Han, K., Wang, Y., Tian, Q., Guo, J., Xu, C., Xu, C.: Ghostnet: more features from cheap operations. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2020). pp. 1577–1586 (2020). https://doi.org/10.1109/CVPR42600.2020.00165

  11. Wang, L., Huang, X., Zheng, Z., Ruan, H.: Surface defect detection method for electronic panels based on double branching and decoupling head structure. PLoS One 18(2), 1–19 (2023). https://doi.org/10.1371/journal.pone.0279035

    Article  CAS  Google Scholar 

  12. Mehta, S., Rastegari, M., Caspi, A., Shapiro, L., Hajishirzi, H.: Espnet: efficient spatial pyramid of dilated convolutions for semantic segmentation. In: Computer Vision - ECCV 2018: 15th European Conference, Munich, Germany, September 8-14, 2018, Proceedings, Part X. Springer-Verlag, pp. 561–580 (2018). https://doi.org/10.1007/978-3-030-01249-6_34

  13. Zhang, W., Ding, X., Liu, Y., Qiao, B.: Slide deep reinforcement learning networks: application for left ventricle segmentation. Pattern Recogn. 141, 109667 (2023). https://doi.org/10.1016/j.patcog.2023.109667

    Article  Google Scholar 

  14. Vigneault, D.M., Xie, W., Ho, C.Y., Bluemke, D.A., Noble, J.A.: Omega-net: fully automatic, multi-view cardiac mr detection, orientation, and segmentation with deep neural networks. Med. Image Anal. 48, 95–106 (2018). https://doi.org/10.1016/j.media.2018.05.008

    Article  PubMed  PubMed Central  Google Scholar 

  15. Yu, C., Wang, J., Peng, C., Gao, C., Yu, G., Sang, N.: Bisenet: Bilateral segmentation network for real-time semantic segmentation. In: Computer Vision – ECCV 2018. Springer International Publishing, pp. 334–349 (2018)

  16. Zhao, H., Qi, X., Shen, X., Shi, J., Jia, J.: Icnet for real-time semantic segmentation on high-resolution images. In: Computer Vision – ECCV 2018. Springer International Publishing, pp. 418–434 (2018)

  17. Paszke, A., Chaurasia, A., Kim, S., Culurciello, E.: Enet: A deep neural network architecture for real-time semantic segmentation. ArXiv. arxiv:1606.02147 (2016)

  18. Ioffe, S., Szegedy, C.: Batch normalization: Accelerating deep network training by reducing internal covariate shift. In: Proceedings of the 32nd International Conference on Machine Learning, Proceedings of Machine Learning Research, vol. 37. PMLR, pp. 448–456 (2015)

  19. Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, (2018). pp. 7132–7141 (2018). https://doi.org/10.1109/CVPR.2018.00745

  20. Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015. Springer International Publishing, pp. 234–241 (2015)

  21. Zhu, W., Liang, R., Yang, J., Cao, Y., Fu, G., Cao, Y.: A sub-region unet for weak defects segmentation with global information and mask-aware loss. Eng. Appl. Artif. Intell. 122, 106011 (2023)

    Article  Google Scholar 

  22. Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Opt. Lasers Eng. 128, 106000 (2020)

    Article  Google Scholar 

  23. Huang, Y., Qiu, C., Guo, Y., Wang, X., Yuan, K.: Surface defect saliency of magnetic tile. Vis. Comput. 36, 85–96 (2018)

    Article  Google Scholar 

  24. Zhang, J., Ding, R., Ban, M., Guo, T.: Fdsnet: an accurate real-time surface defect segmentation network. In: ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). pp. 3803–3807 (2022)

  25. Shrivastava, A., Gupta, A., Girshick, R.: Training region-based object detectors with online hard example mining (2016)

  26. Kingma, D., Ba, J.: Adam: A method for stochastic optimization. In: International Conference on Learning Representations (ICLR). San Diega, CA, USA, (2015)

  27. Zhao, H., Shi, J., Qi, X., Wang, X., Jia, J.: Pyramid scene parsing network. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp. 6230–6239 (2017). https://doi.org/10.1109/CVPR.2017.660

  28. Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Kopf, A., Yang, E., DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J., Chintala, S.: In: Advances in Neural Information Processing Systems 3. Curran Associates, Inc., pp. 8024–8035 (2019)

  29. Dong, H., Song, K., He, Y., Xu, J., Yan, Y., Meng, Q.: Pga-net: Pyramid feature fusion and global context attention network for automated surface defect detection. IEEE Trans. Industr. Inf. 16(12), 7448–7458 (2020). https://doi.org/10.1109/TII.2019.2958826

    Article  Google Scholar 

  30. Chen, X., Fu, C., Tie, M., Sham, C.W., Ma, H.: Affnet: an attention-based feature-fused network for surface defect segmentation. Appl. Sci. (2023). https://doi.org/10.3390/app13116428

    Article  Google Scholar 

  31. Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp. 3431–3440 (2015). https://doi.org/10.1109/CVPR.2015.7298965

  32. Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Trans. Pattern Anal. Mach. Intell. 40(4), 834–848 (2018). https://doi.org/10.1109/TPAMI.2017.2699184

    Article  PubMed  Google Scholar 

  33. Qin, X., Zhang, Z., Huang, C., Dehghan, M., Zaiane, O.R., Jagersand, M.: U2-net: going deeper with nested u-structure for salient object detection. Pattern Recogn. 106, 107404 (2020)

    Article  Google Scholar 

  34. Wang, Y., Zhou, Q., Liu, J., Xiong, J., Gao, G., Wu, X., Latecki, L.J.: Lednet: a lightweight encoder-decoder network for real-time semantic segmentation. In: 2019 IEEE International Conference on Image Processing (ICIP). pp. 1860–1864 (2019)

  35. Poudel, R.P.K., Liwicki, S., Cipolla, R.: Fast-scnn: fast semantic segmentation network. In: 30th British Machine Vision Conference (2019)

  36. Jiang, X., Wang, H., Chen, Y., Wu, Z., Wang, L., Zou, B., Yang, Y., Cui, Z., Cai, Y., Yu, T., Lv, C., Wu, Z.: Mnn: a universal and efficient inference engine. In: MLSys (2020)

Download references

Author information

Authors and Affiliations

  1. Department of Electrical and Information Engineering, Faculty of Engineering, Universitas Gadjah Mada, Jl. Grafika No. 2, Bulaksumur, Yogyakarta, 55281, Indonesia

    Igi Ardiyanto

Authors
  1. Igi Ardiyanto
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Igi Ardiyanto developed and implemented the algorithms for the work.

Corresponding author

Correspondence to Igi Ardiyanto.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ardiyanto, I. Edge devices-oriented surface defect segmentation by GhostNet Fusion Block and Global Auxiliary Layer. J Real-Time Image Proc 21, 13 (2024). https://doi.org/10.1007/s11554-023-01394-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11554-023-01394-5

Keywords

Search

Navigation