Skip to main content

Segmentation-based deep-learning approach for surface-defect detection

Abstract

Automated surface-anomaly detection using machine learning has become an interesting and promising area of research, with a very high and direct impact on the application domain of visual inspection. Deep-learning methods have become the most suitable approaches for this task. They allow the inspection system to learn to detect the surface anomaly by simply showing it a number of exemplar images. This paper presents a segmentation-based deep-learning architecture that is designed for the detection and segmentation of surface anomalies and is demonstrated on a specific domain of surface-crack detection. The design of the architecture enables the model to be trained using a small number of samples, which is an important requirement for practical applications. The proposed model is compared with the related deep-learning methods, including the state-of-the-art commercial software, showing that the proposed approach outperforms the related methods on the specific domain of surface-crack detection. The large number of experiments also shed light on the required precision of the annotation, the number of required training samples and on the required computational cost. Experiments are performed on a newly created dataset based on a real-world quality control case and demonstrates that the proposed approach is able to learn on a small number of defected surfaces, using only approximately 25–30 defective training samples, instead of hundreds or thousands, which is usually the case in deep-learning applications. This makes the deep-learning method practical for use in industry where the number of available defective samples is limited. The dataset is also made publicly available to encourage the development and evaluation of new methods for surface-defect detection.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Notes

  1. 1.

    The Kolektor surface-defect dataset is publicly available at http://www.vicos.si/Downloads/KolektorSDD.

References

  1. Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., et al. (2015). TensorFlow: Large-scale machine learning on heterogeneous systems. https://www.tensorflow.org/.

  2. Bulnes, F. G., Usamentiaga, R., Garcia, D. F., & Molleda, J. (2016). An efficient method for defect detection during the manufacturing of web materials. Journal of Intelligent Manufacturing, 27(2), 431–445. https://doi.org/10.1007/s10845-014-0876-9.

    Article  Google Scholar 

  3. Chen, P. H., & Ho, S. S. (2016). Is overfeat useful for image-based surface defect classification tasks? In IEEE international conference on image processing (ICIP) (pp. 749–753).

  4. Chen, L. C., Zhu, Y., Papandreou, G., Schroff, F., & Adam, H. (2018). Encoder–Decoder with atrous separable convolution for semantic image segmentation. Tech. rep.

  5. Chollet, F. (2017). Xception: Deep learning with depthwise separable convolutions. Computer Vision and Pattern Recognition, 2017, 1800–1807. https://doi.org/10.1109/CVPR.2017.195.

    Article  Google Scholar 

  6. Cognex. (2018). VISIONPRO VIDI: Deep learning-based software for industrial image analysis. https://www.cognex.com/products/machine-vision/vision-software/visionpro-vidi

  7. Faghih-Roohi, S., Hajizadeh, S., Núñez, A., Babuska, R., & Schutter, B. D. (2016). Deep convolutional neural networks for detection of rail surface defects deep convolutional neural networks for detection of rail surface defects. In International joint conference on neural networks (pp. 2584–2589).

  8. Kaiming, H., Gkioxara, G., Dollar, P., & Girshick, R. (2017). Mask R-CNN. In ICCV (pp. 2961–2969).

  9. Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). ImageNet classification with deep convolutional neural networks. In Advances in neural information processing systems (Vol. 25, pp. 1097–1105).

  10. Kuo, C. F. J., Hsu, C. T. M., Liu, Z. X., & Wu, H. C. (2014). Automatic inspection system of LED chip using two-stages back-propagation neural network. Journal of Intelligent Manufacturing, 25(6), 1235–1243. https://doi.org/10.1007/s10845-012-0725-7.

    Article  Google Scholar 

  11. Lin, H., Li, B., Wang, X., Shu, Y., & Niu, S. (2018). Automated defect inspection of LED chip using deep convolutional neural network. Journal of Intelligent Manufacturing, 1–10. https://doi.org/10.1007/s10845-018-1415-x.

    Article  Google Scholar 

  12. Lin, T. Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014). Microsoft COCO: Common objects in context. LNCS 8693 LNCS(PART 5):740–755.

  13. Long, J., Shelhamer, E., & Darrell, T. (2015). Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition (Vol. 8828, pp. 3431–3440). https://doi.org/10.1109/CVPR.2015.7298965.

  14. Masci, J., Meier, U., Ciresan, D., Schmidhuber, J., & Fricout, G. (2012). Steel defect classification with Max-Pooling Convolutional Neural Networks. In Proceedings of the international joint conference on neural networks. https://doi.org/10.1109/IJCNN.2012.6252468.

  15. Oztemel, E., & Gursev, S. (2018). Literature review of Industry 4.0 and related technologies. Journal of Intelligent Manufacturing. https://doi.org/10.1007/s10845-018-1433-8.

    Article  Google Scholar 

  16. Paniagua, B., Vega-Rodríguez, M. A., Gomez-Pulido, J. A., & Sanchez-Perez, J. M. (2010). Improving the industrial classification of cork stoppers by using image processing and Neuro-Fuzzy computing. Journal of Intelligent Manufacturing, 21(6), 745–760. https://doi.org/10.1007/s10845-009-0251-4.

    Article  Google Scholar 

  17. Rački, D., Tomaževič, D., & Skočaj, D. (2018). A compact convolutional neural network for textured surface anomaly detection. In IEEE winter conference on applications of computer vision (pp. 1331–1339). https://doi.org/10.1109/WACV.2018.00150.

  18. Ronneberger, O., Fischer, P., & Brox, T. (2015). U-Net: Convolutional networks for biomedical image segmentation. In Medical image computing and computer-assisted intervention—MICCAI 2015 (pp. 234–241).

  19. Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., et al. (2015). ImageNet large scale visual recognition challenge. International Journal of Computer Vision, 115(3), 211–252. https://doi.org/10.1007/s11263-015-0816-y.

    Article  Google Scholar 

  20. Sermanet, P., & Eigen, D. (2014). OverFeat : Integrated recognition, localization and detection using convolutional networks. In International conference on learning representations (ICLR2014), CBLS.

  21. Weimer, D., Scholz-Reiter, B., & Shpitalni, M. (2016). Design of deep convolutional neural network architectures for automated feature extraction in industrial inspection. CIRP Annals-Manufacturing Technology, 65(1), 417–420. https://doi.org/10.1016/j.cirp.2016.04.072.

    Article  Google Scholar 

  22. Weimer, D., Thamer, H., & Scholz-Reiter, B. (2013). Learning defect classifiers for textured surfaces using neural networks and statistical feature representations. Procedia CIRP, 7, 347–352. https://doi.org/10.1016/j.procir.2013.05.059.

    Article  Google Scholar 

  23. Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., & Torralba, A. (2016). Learning deep features for discriminative localization. In Computer vision and pattern recognition.

Download references

Acknowledgements

This work was supported in part by the following research projects and programs: GOSTOP program C3330-16-529000 co-financed by the Republic of Slovenia and the European Regional Development Fund, ARRS research project J2-9433 (DIVID), and ARRS research programme P2-0214. We would also like to thank the company Kolektor Orodjarna d. o. o. for providing images for the proposed dataset as well as for providing high quality annotations.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Domen Tabernik.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Tabernik, D., Šela, S., Skvarč, J. et al. Segmentation-based deep-learning approach for surface-defect detection. J Intell Manuf 31, 759–776 (2020). https://doi.org/10.1007/s10845-019-01476-x

Download citation

Keywords

  • Surface-defect detection
  • Visual inspection
  • Quality control
  • Deep learning
  • Computer vision
  • Segmentation networks
  • Industry 4.0