Abstract
Automated surface-anomaly detection using machine learning has become an interesting and promising area of research, with a very high and direct impact on the application domain of visual inspection. Deep-learning methods have become the most suitable approaches for this task. They allow the inspection system to learn to detect the surface anomaly by simply showing it a number of exemplar images. This paper presents a segmentation-based deep-learning architecture that is designed for the detection and segmentation of surface anomalies and is demonstrated on a specific domain of surface-crack detection. The design of the architecture enables the model to be trained using a small number of samples, which is an important requirement for practical applications. The proposed model is compared with the related deep-learning methods, including the state-of-the-art commercial software, showing that the proposed approach outperforms the related methods on the specific domain of surface-crack detection. The large number of experiments also shed light on the required precision of the annotation, the number of required training samples and on the required computational cost. Experiments are performed on a newly created dataset based on a real-world quality control case and demonstrates that the proposed approach is able to learn on a small number of defected surfaces, using only approximately 25–30 defective training samples, instead of hundreds or thousands, which is usually the case in deep-learning applications. This makes the deep-learning method practical for use in industry where the number of available defective samples is limited. The dataset is also made publicly available to encourage the development and evaluation of new methods for surface-defect detection.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
The Kolektor surface-defect dataset is publicly available at http://www.vicos.si/Downloads/KolektorSDD.
References
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., et al. (2015). TensorFlow: Large-scale machine learning on heterogeneous systems. https://www.tensorflow.org/.
Bulnes, F. G., Usamentiaga, R., Garcia, D. F., & Molleda, J. (2016). An efficient method for defect detection during the manufacturing of web materials. Journal of Intelligent Manufacturing, 27(2), 431–445. https://doi.org/10.1007/s10845-014-0876-9.
Chen, P. H., & Ho, S. S. (2016). Is overfeat useful for image-based surface defect classification tasks? In IEEE international conference on image processing (ICIP) (pp. 749–753).
Chen, L. C., Zhu, Y., Papandreou, G., Schroff, F., & Adam, H. (2018). Encoder–Decoder with atrous separable convolution for semantic image segmentation. Tech. rep.
Chollet, F. (2017). Xception: Deep learning with depthwise separable convolutions. Computer Vision and Pattern Recognition, 2017, 1800–1807. https://doi.org/10.1109/CVPR.2017.195.
Cognex. (2018). VISIONPRO VIDI: Deep learning-based software for industrial image analysis. https://www.cognex.com/products/machine-vision/vision-software/visionpro-vidi
Faghih-Roohi, S., Hajizadeh, S., Núñez, A., Babuska, R., & Schutter, B. D. (2016). Deep convolutional neural networks for detection of rail surface defects deep convolutional neural networks for detection of rail surface defects. In International joint conference on neural networks (pp. 2584–2589).
Kaiming, H., Gkioxara, G., Dollar, P., & Girshick, R. (2017). Mask R-CNN. In ICCV (pp. 2961–2969).
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). ImageNet classification with deep convolutional neural networks. In Advances in neural information processing systems (Vol. 25, pp. 1097–1105).
Kuo, C. F. J., Hsu, C. T. M., Liu, Z. X., & Wu, H. C. (2014). Automatic inspection system of LED chip using two-stages back-propagation neural network. Journal of Intelligent Manufacturing, 25(6), 1235–1243. https://doi.org/10.1007/s10845-012-0725-7.
Lin, H., Li, B., Wang, X., Shu, Y., & Niu, S. (2018). Automated defect inspection of LED chip using deep convolutional neural network. Journal of Intelligent Manufacturing, 1–10. https://doi.org/10.1007/s10845-018-1415-x.
Lin, T. Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014). Microsoft COCO: Common objects in context. LNCS 8693 LNCS(PART 5):740–755.
Long, J., Shelhamer, E., & Darrell, T. (2015). Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition (Vol. 8828, pp. 3431–3440). https://doi.org/10.1109/CVPR.2015.7298965.
Masci, J., Meier, U., Ciresan, D., Schmidhuber, J., & Fricout, G. (2012). Steel defect classification with Max-Pooling Convolutional Neural Networks. In Proceedings of the international joint conference on neural networks. https://doi.org/10.1109/IJCNN.2012.6252468.
Oztemel, E., & Gursev, S. (2018). Literature review of Industry 4.0 and related technologies. Journal of Intelligent Manufacturing. https://doi.org/10.1007/s10845-018-1433-8.
Paniagua, B., Vega-Rodríguez, M. A., Gomez-Pulido, J. A., & Sanchez-Perez, J. M. (2010). Improving the industrial classification of cork stoppers by using image processing and Neuro-Fuzzy computing. Journal of Intelligent Manufacturing, 21(6), 745–760. https://doi.org/10.1007/s10845-009-0251-4.
Rački, D., Tomaževič, D., & Skočaj, D. (2018). A compact convolutional neural network for textured surface anomaly detection. In IEEE winter conference on applications of computer vision (pp. 1331–1339). https://doi.org/10.1109/WACV.2018.00150.
Ronneberger, O., Fischer, P., & Brox, T. (2015). U-Net: Convolutional networks for biomedical image segmentation. In Medical image computing and computer-assisted intervention—MICCAI 2015 (pp. 234–241).
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., et al. (2015). ImageNet large scale visual recognition challenge. International Journal of Computer Vision, 115(3), 211–252. https://doi.org/10.1007/s11263-015-0816-y.
Sermanet, P., & Eigen, D. (2014). OverFeat : Integrated recognition, localization and detection using convolutional networks. In International conference on learning representations (ICLR2014), CBLS.
Weimer, D., Scholz-Reiter, B., & Shpitalni, M. (2016). Design of deep convolutional neural network architectures for automated feature extraction in industrial inspection. CIRP Annals-Manufacturing Technology, 65(1), 417–420. https://doi.org/10.1016/j.cirp.2016.04.072.
Weimer, D., Thamer, H., & Scholz-Reiter, B. (2013). Learning defect classifiers for textured surfaces using neural networks and statistical feature representations. Procedia CIRP, 7, 347–352. https://doi.org/10.1016/j.procir.2013.05.059.
Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., & Torralba, A. (2016). Learning deep features for discriminative localization. In Computer vision and pattern recognition.
Acknowledgements
This work was supported in part by the following research projects and programs: GOSTOP program C3330-16-529000 co-financed by the Republic of Slovenia and the European Regional Development Fund, ARRS research project J2-9433 (DIVID), and ARRS research programme P2-0214. We would also like to thank the company Kolektor Orodjarna d. o. o. for providing images for the proposed dataset as well as for providing high quality annotations.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Tabernik, D., Šela, S., Skvarč, J. et al. Segmentation-based deep-learning approach for surface-defect detection. J Intell Manuf 31, 759–776 (2020). https://doi.org/10.1007/s10845-019-01476-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10845-019-01476-x