Abstract
Defects that appear on a leather surface may be the result of natural variations or poor handling during the manufacturing process. Visual inspection in the factory is one of the essential steps in the process of quality assurance. It should be done before the finished products are being dispatched to the customer. Thus far, the detection of the leather defects is still carried out manually, which is labour intensive, tedious, and might be liable to human error. Therefore, in this paper, we propose an automatic leather defect localization and detection system by employing a series of digital image processing methods based on deep learning. Succinctly, a convolutional neural network (CNN) is utilized to perform the detection task, that is to determine the presence of the defect on a leather patch. Then, the detected defective leather patch is processed to the localization operation, which is to identify the boundary coordinates in pixel level. For the detection task, the result achieved using AlexNet as the feature descriptor and SVM as the classifier is 100%. For the localization stage, we have demonstrated that the instance segmentation technique, Faster R-CNN outperforms the YOLOv2 by obtaining the Intersection over Union (IoU) of 73%. In addition, extensive experiments and comparisons of the state-of-the-art approaches are presented to verify the effectiveness of the proposed algorithms.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Aslam M, Khan TM, Naqvi SS, Holmes G, Naffa R (2020) Ensemble convolutional neural networks with knowledge transfer for leather defect classification in industrial settings. IEEE Access 8:198600–198614
Bong H-Q, Truong Q-B, Nguyen H-C, Nguyen M-T (2018) Vision-based inspection system for leather surface defect detection and classification. In: 2018 5th NAFOSTED Conference on Information and Computer Science (NICS), IEEE, pp 300–304
Deng J, Dong W, Socher R, Li L-J, Li K, Fei-Fei L (2009) Imagenet: a large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition. IEEE, pp 248–255
Deng J, Liu J, Wu C, Zhong T, Gu G, Ling BW-K (2020) A novel framework for classifying leather surface defects based on a parameter optimized residual network. IEEE Access 8:192109–192118
Everingham M, Van Gool L, Williams CK, Winn J, Zisserman A (2010) The pascal visual object classes (voc) challenge. Int J Comput Vis 88(2):303–338
Girshick R (2015) Fast r-cnn. In: Proceedings of the IEEE international conference on computer vision, pp 1440–1448
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778
Huang G, Liu Z, Van Der Maaten L, Weinberger K Q (2017) Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4700–4708
Iandola F N, Han S, Moskewicz MW, Ashraf K, Dally WJ, Keutzer K (2016) Sequeezenet: Alexnet-level accuracy with 50x fewer parameters and< 0.5 mb model size. arXiv preprint arXiv:1602.07360
Jain AK, Duin RPW, Mao J (2000) Statistical pattern recognition: a review. IEEE Trans Pattern Anal Mach Intell 22(1):4–37
Jawahar M, Babu NC, Vani K (2014) Leather texture classification using wavelet feature extraction technique. In: 2014 IEEE International Conference on Computational Intelligence and Computing Research, IEEE, pp 1–4
Jawahar M, Babu NC, Vani K, Anbarasi LJ, Geetha S (2021) Vision based inspection system for leather surface defect detection using fast convergence particle swarm optimization ensemble classifier approach. Multimed Tools Appl 80(3):4203–4235
Jian L, Wei H, Bin H (2010) Research on inspection and classification of leather surface defects based on neural network and decision tree. In: 2010 International Conference On Computer Design and Applications, Vol 2, IEEE, pp V2–381
Kasi MK, Rao JB, Sahu VK (2014) Identification of leather defects using an autoadaptive edge detection image processing algorithm. In: 2014 International Conference on High Performance Computing and Applications (ICHPCA), IEEE, pp 1–4
Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105
Kwak C, Ventura JA, Tofang-Sazi K (2000) A neural network approach for defect identification and classification on leather fabric. J Intell Manuf 11(5):485–499
LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324
Lin T-Y, Maire M, Belongie S, Hays J, Perona P, Ramanan D, Dollár P, Zitnick CL (2014) Microsoft coco: common objects in context. In: European conference on computer vision. Springer, pp 740–755
Liong S-T, Gan Y, Huang Y-C, Liu K-H, Yau W-C (2019a) Integrated neural network and machine vision approach for leather defect classification. arXiv preprint arXiv:1905.11731
Liong S-T, Gan Y, Huang Y-C, Yuan C-A, Chang H-C (2019b) Automatic defect segmentation on leather with deep learning. arXiv preprint arXiv:1903.12139
Mundt M, Majumder S, Murali S, Panetsos P, Ramesh V (2019) Meta-learning convolutional neural architectures for multi-target concrete defect classification with the concrete defect bridge image dataset. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 11196–11205
Pistori H, Paraguassu WA, Martins PS, Conti MP, Pereira MA, Jacinto MA (2018) Defect detection in raw hide and wet blue leather. In: Computational Modelling of Objects Represented in Images. Fundamentals, Methods and Applications: Proceedings of the International Symposium CompIMAGE 2006 (Coimbra, Portugal, 20–21 October 2006). CRC Press, p 355
Ren S, He K, Girshick R, Sun J (2016) Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 39(6):1137–1149
Ren S, He K, Girshick R, Sun J(2015) , Faster r-cnn: towards real-time object detection with region proposal networks. In: Advances in neural information processing systems, pp 91–99
Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D (2017) Grad-cam: visual explanations from deep networks via gradient-based localization. In: Proceedings of the IEEE International Conference on Computer Vision, pp 618–626
Serafim AL (1992) Object detection in images of natural scenes represented by ar models using laplacian pyramids: application to leather defects localization. In: Proceedings of the 1992 International Conference on Industrial Electronics, Control, Instrumentation, and Automation, IEEE, pp 716–720
Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556
Statista (2019) Luxury leather goods. https://www.statista.com
Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1–9
Tschandl P, Rosendahl C, Kittler H (2018) The ham10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Sci Data 5(1):1–9
Ventura JA, Chen J-M (1996) A structural model for shape recognition using neural nets. J Intell Manuf 7(1):1–11
Villar P, Mora M, Gonzalez P (2011) A new approach for wet blue leather defect segmentation. In: Iberoamerican Congress on Pattern Recognition. Springer, pp 591–598
Winiarti S, Prahara A, Murinto DPI (2018) Pre-trained convolutional neural network for classification of tanning leather image. Network (CNN) 9(1)
Zhang X, Zhou X, Lin M, Sun J (2018) Shufflenet: an extremely efficient convolutional neural network for mobile devices. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 6848–6856
Acknowledgements
This work was funded by Ministry of Science and Technology (MOST), Taiwan (Grant No. MOST 109-2221-E-035-065-MY2, 109-2218-E-035 -002- and 108-2218-E-035-018-).
Author information
Authors and Affiliations
Corresponding authors
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Gan, Y.S., Liong, ST., Zheng, D. et al. Detection and localization of defects on natural leather surfaces. J Ambient Intell Human Comput 14, 1785–1799 (2023). https://doi.org/10.1007/s12652-021-03396-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12652-021-03396-2