Skip to main content
SpringerLink
Log in

Recent advances in surface defect inspection of industrial products using deep learning techniques

  • Survey Paper
  • Published:
The International Journal of Advanced Manufacturing Technology Aims and scope Submit manuscript

Abstract

Manual surface inspection methods performed by quality inspectors do not satisfy the continuously increasing quality standards of industrial manufacturing processes. Machine vision provides a solution by using an automated visual inspection (AVI) system to perform quality inspection and remove defective products. Numerous studies and works have been conducted on surface inspection algorithms. With the advent of deep learning, a number of new algorithms have been developed for better inspection. In this paper, the state-of-the-art in surface defect inspection using deep learning is presented. In particular, we focus on the inspection of industrial products in semiconductor, steel, and fabric manufacturing processes. This work makes three contributions. First, we present the prior literature reviews on vision-based surface defect inspection and analyze the recent AVI-related hardware and software. Second, we review traditional surface defect inspection algorithms including statistical methods, spectral methods, model-based methods, and learning-based methods. Third, we investigate recent advances in deep learning-based inspection algorithms and present their applications in the steel, fabric, and semiconductor industries. Furthermore, we provide information on publicly available datasets containing surface image samples to facilitate the research on deep learning-based surface inspection.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Data availability

Not applicable.

References

  1. Steger C, Ulrich M, Wiedemann C (2018) Machine vision algorithms and applications: second completely revised and Enlarged Edition. Wiley-VCH, Hoboken

  2. Hornberg A (2017) Handbook of machine and computer vision: the guide for developers and users, Second edn. Wiley-VCH. https://doi.org/10.1002/9783527413409

  3. Sun XH, Gu JA, Tang SX, Li J (2018) Research progress of visual inspection technology of steel products-a review. Appl Sci-Basel 8(11). https://doi.org/10.3390/app8112195

  4. Golnabi H, Asadpour A (2007) Design and application of industrial machine vision systems. Robot Comput Integr Manuf 23(6):630–637. https://doi.org/10.1016/j.rcim.2007.02.005

    Article  Google Scholar 

  5. Ozseven T (2019) Surface defect detection and quantification with image processing methods. In: Ozseven T (ed) Theoretical investigations and applied studies in engineering. Ekin Publishing House, pp 63–98

  6. Newman TS, Jain AK (1995) A survey of automated visual inspection. Comput Vis Image Underst 61(2):231–262

    Article  Google Scholar 

  7. Neogi N, Mohanta DK, Dutta PK (2014) Review of vision-based steel surface inspection systems. EURASIP J Image Vide:1–19. https://doi.org/10.1186/1687-5281-2014-50

  8. Gao C, Zhou J, Wong WK, Gao T Woven fabric defect detection based on convolutional neural network for binary classification. In: Artificial Intelligence on Fashion and Textiles Conference, AIFT 2018, June 27, 2018 - June 29, 2018, Hong Kong, China, 2019. Advances in intelligent systems and computing. Springer Verlag, pp 307–313. https://doi.org/10.1007/978-3-319-99695-0_37

  9. Huang SH, Pan YC (2015) Automated visual inspection in the semiconductor industry: a survey. Comput Ind 66:1–10

    Article  Google Scholar 

  10. Malamas EN, Petrakis EGM, Zervakis M, Petit L, Legat JD (2003) A survey on industrial vision systems, applications and tools. Image Vis Comput 21(2):171–188. https://doi.org/10.1016/S0262-8856(02)00152-X

    Article  Google Scholar 

  11. Xie X (2008) A review of recent advances in surface defect detection using texture analysis techniques. Electron Lett Comput Vis Image Anal 7(3):1–22

    MathSciNet  Google Scholar 

  12. Kumar A (2008) Computer-vision-based fabric defect detection: a survey. IEEE Trans Ind Electron 55(1):348–363. https://doi.org/10.1109/Tie.2007.896476

    Article  Google Scholar 

  13. Mahajan PM, Kolhe SR, Patil PM (2009) A review of automatic fabric defect detection techniques. Adv Comput Res 1(2):18–29

    Google Scholar 

  14. Hani AFM, Malik AS, Kamil R, Thong CM (2012) A review of SMD-PCB defects and detection algorithms. Proc SPIE 8350. https://doi.org/10.1117/12.920531

  15. Ngan HYT, Pang GKH, Yung NHC (2011) Automated fabric defect detection-a review. Image Vis Comput 29(7):442–458. https://doi.org/10.1016/j.imavis.2011.02.002

    Article  Google Scholar 

  16. Hanbay K, Talu MF, Ozguven OF (2016) Fabric defect detection systems and methods-a systematic literature review. Optik 127(24):11960–11973. https://doi.org/10.1016/j.ijleo.2016.09.110

    Article  Google Scholar 

  17. Anitha DB, Rao M (2017) A survey on defect detection in bare PCB and assembled PCB using image processing techniques. In: 2017 2nd Ieee international conference on wireless communications, signal processing and networking (Wispnet), pp 39–43. https://doi.org/10.1109/WiSPNET.2017.8299715

  18. Lu R, Wu A, Zhang T, Wang Y (2018) Review on automated optical (visual) inspection and its application in defect detection. Acta Opt Sin 38(437 (8)):15–50

    Google Scholar 

  19. Shirvaikar M (2006) Trends in automated visual inspection. J Real Time Image Process 1(1):41–43. https://doi.org/10.1007/s11554-006-0009-6

    Article  Google Scholar 

  20. Shreya SR, Priya CS, Rajeshware GS (2017) Design of machine vision system for high speed manufacturing environments. In: India Conference, 2017

    Google Scholar 

  21. OpenCV Tutorials. https://docs.opencv.org/master/d9/df8/tutorial_root.html. Accessed Oct. 2019

  22. HALCON_18.11_brochure. https://www.mvtec.com. Accessed Oct. 2019

  23. VisionPro. https://www.cognex.com. Accessed Oct. 2019

  24. Demant C, Streicher-Abel B, Garnica C (2013) Industrial image processing: visual quality control in manufacturing, 2nd edn. Springer. https://doi.org/10.1007/978-3-642-33905-9

  25. Van Gool L, Wambacq P, Oosterlinck A (1991) Intelligent robotic vision systems. Marcel Dekker Inc, New York

    Google Scholar 

  26. Bible RE (1984) Automated optical inspection of printed circuit boards. Test Meas World Oct.:208–213

  27. Moganti M, Ercal F, Dagli CH, Shou T (1996) Automatic PCB inspection algorithms: a survey. Comput Vis Image Underst 63(2):287–313

    Article  Google Scholar 

  28. Silven O, Virtanen I, Pietikainen M (1985) Cad data-based comparison method for printed wiring board (PWB) inspection. In: Society of Photo-optical Instrumentation Engineers Conference Series, 17 January 1985. https://doi.org/10.1117/12.946210

  29. Li YD, Zhao WG, Pan JH (2017) Deformable patterned fabric defect detection with fisher criterion-based deep learning. IEEE Trans Autom Sci Eng 14(2):1256–1264. https://doi.org/10.1109/Tase.2016.2520955

    Article  Google Scholar 

  30. Liu K, Wang H, Chen H, Qu E, Sun H (2017) Steel surface defect detection using a new Haar-Weibull-variance model in unsupervised manner. IEEE Trans Instrum Meas 99:1–12

    Google Scholar 

  31. Huangpeng Q, Zhang H, Zeng XR, Huang WW (2018) Automatic visual defect detection using texture prior and low-rank representation. IEEE Access 6:37965–37976. https://doi.org/10.1109/Access.2018.2852663

    Article  Google Scholar 

  32. Haralick RM, Shanmugam K, Dinstein I’H (1973) Textural features for image classification. IEEE Trans Syst Man Cybern SMC-3(6):610–621. https://doi.org/10.1109/TSMC.1973.4309314

    Article  Google Scholar 

  33. Ojala T, Harwood I (1996) A comparative study of texture measures with classification based on feature distributions. Pattern Recogn 29(1):51–59

    Article  Google Scholar 

  34. Tajeripour F, Kabir E, Sheikhi A (2008) Fabric defect detection using modified local binary patterns. EURASIP J Adv Sig Process 2008. https://doi.org/10.1155/2008/783898

  35. Tang B, Kong J, Wu S (2017) Review of surface defect detection based on machine vision. J Chin Image Graph 22(12):1640–1663. https://doi.org/10.11834/jig.160623

    Article  Google Scholar 

  36. Ashour MW, Khalid F, Halin AA, Abdullah LN, Darwish SH (2018) Surface defects classification of hot-rolled steel strips using multi-directional shearlet features. Arabian Journal for Science & Engineering 44:2925–2932. https://doi.org/10.1007/s13369-018-3329-5

  37. Luo Q, Sun Y, Li P, Simpson O, He Y (2018) Generalized completed local binary patterns for time-efficient steel surface defect classification. IEEE Trans Instrum Meas 99:1–13

    Google Scholar 

  38. Li M, Wan SH, Deng ZM, Wang YJ (2019) Fabric defect detection based on saliency histogram features. Comput Intell-Us 35(3):517–534. https://doi.org/10.1111/coin.12206

    Article  MathSciNet  Google Scholar 

  39. Luo Q, Fang X, Sun Y, Liu L, Simpson O (2019) Surface defect classification for hot-rolled steel strips by selectively dominant local binary patterns. IEEE Access 99:1–1

    Article  Google Scholar 

  40. Li WC, Tsai DM (2012) Wavelet-based defect detection in solar wafer images with inhomogeneous texture. Pattern Recogn 45(2):742–756. https://doi.org/10.1016/j.patcog.2011.07.025

    Article  Google Scholar 

  41. Malek AS, Drean JY, Bigue L, Osselin JF (2013) Optimization of automated online fabric inspection by fast Fourier transform (FFT) and cross-correlation. Text Res J 83(3):256–268. https://doi.org/10.1177/0040517512458340

    Article  Google Scholar 

  42. Bissi L, Baruffa G, Placidi P, Ricci E, Scorzoni A, Valigi P (2013) Automated defect detection in uniform and structured fabrics using Gabor filters and PCA. J Vis Commun Image Represent 24(7):838–845

    Article  Google Scholar 

  43. Hu GH, Zhang GH, Wang QH (2014) Automated defect detection in textured materials using wavelet-domain hidden Markov models. Opt Eng 53(9):093107

    Article  Google Scholar 

  44. Wen ZJ, Cao JJ, Liu XP, Ying SH (2014) Fabric defects detection using adaptive wavelets. Int J Cloth Sci Technol 26(3):202–211. https://doi.org/10.1108/Ijcst-03-2013-0031

    Article  Google Scholar 

  45. Hu GH, Wang QH, Zhang GH (2015) Unsupervised defect detection in textiles based on Fourier analysis and wavelet shrinkage. Appl Opt 54(10):2963–2980. https://doi.org/10.1364/Ao.54.002963

    Article  Google Scholar 

  46. Bi X, Xu XP, Shen JH (2015) An automatic detection method of Mura defects for liquid crystal display using real Gabor filters. In: 2015 8th International Congress on Image and Signal Processing (Cisp), pp 871–875. https://doi.org/10.1109/CISP.2015.7408000

  47. Hu GH (2015) Automated defect detection in textured surfaces using optimal elliptical Gabor filters. Optik 126(14):1331–1340. https://doi.org/10.1016/j.ijleo.2015.04.017

    Article  Google Scholar 

  48. Tong L, Wong WK, Kwong CK (2016) Differential evolution-based optimal Gabor filter model for fabric inspection. Neurocomputing 173:1386–1401. https://doi.org/10.1016/j.neucom.2015.09.011

    Article  Google Scholar 

  49. Chol DC, Jeon YJ, Kim SH, Moon S, Yun JP, Kim SW (2017) Detection of pinholes in steel slabs using Gabor filter combination and morphological features. ISIJ Int 57(6):1045–1053. https://doi.org/10.2355/isijinternational.ISIJINT-2016-160

    Article  Google Scholar 

  50. Ma JX, Wang YX, Shi C, Lu CW (2018) Fast surface defect detection using improved Gabor filters. In: 2018 25th Ieee International Conference on Image Processing (Icip), pp 1508–1512. https://doi.org/10.1109/ICIP.2018.8451351

  51. Ren RX, Hung T, Tan KC (2018) A generic deep-learning-based approach for automated surface inspection. IEEE Trans Cybern 48(3):929–940. https://doi.org/10.1109/Tcyb.2017.2668395

    Article  Google Scholar 

  52. Kindermann R, Snell JL (1980) Markov random fields and their applications

  53. Comer ML, Delp EJ (1999) Segmentation of textured images using a multiresolution Gaussian autoregressive model. IEEE Trans Image Process 8(3):408–420

    Article  Google Scholar 

  54. Cohen FS, Fan Z, Attali S (1991) Automated inspection of textile fabrics using textural models. IEEE Trans Pattern Anal Mach Intell 13(8):803–808

    Article  Google Scholar 

  55. Xu LJ, Huang Q (2012) Modeling the interactions among neighboring nanostructures for local feature characterization and defect detection. IEEE Trans Autom Sci Eng 9(4):745–754. https://doi.org/10.1109/Tase.2012.2209417

    Article  Google Scholar 

  56. Kulkarni R, Banoth E, Pal P (2019) Automated surface feature detection using fringe projection: an autoregressive modeling-based approach. Opt Lasers Eng 121:506–511. https://doi.org/10.1016/j.optlaseng.2019.05.014

    Article  Google Scholar 

  57. Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297

    MATH  Google Scholar 

  58. Altman NS (1992) An introduction to kernel and nearest-neighbor nonparametric regression. Am Stat 46(3):175–185

    MathSciNet  Google Scholar 

  59. Breiman L, Friedman J, Stone CJ, Olshen RA (1984) Classification and regression trees. CRC Press, Boca Raton

  60. Jia HB, Murphey YL, Shi JJ, Chang TS (2004) An intelligent real-time vision system for surface defect detection. Int C Patt Recog:239–242. doi: https://doi.org/10.1109/Icpr.2004.1334512

  61. Gao XD, Gao B, He Z, Xin WH (2006) Fabric defect detection based on support vector machine. J Text Res 27(5):26–28

    Google Scholar 

  62. Kang SB, Lee JH, Song KY, Pahk HJ (2009) Automatic defect classification of TFT-LCD panels using machine learning. In: 2009 IEEE International Symposium on Industrial Electronics, pp 2175–2177. https://doi.org/10.1109/ISIE.2009.5213760

  63. Baly R, Hajj H (2012) Wafer classification using support vector machines. IEEE Trans Semicond Manuf 25(3):373–383. https://doi.org/10.1109/Tsm.2012.2196058

    Article  Google Scholar 

  64. Huang W, Lu H (2013) Automatic defect classification of TFT-LCD panels with shape, histogram and color features. Int J Image Graph 13(03):1350011. https://doi.org/10.1142/S0219467813500113

  65. Xie LJ, Huang R, Cao ZQ (2013) Detection and classification of defect patterns in optical inspection using support vector machines. Lect Notes Comput Sci 7995:376–384

    Article  Google Scholar 

  66. Zhang ZQ, Wang XD, Liu S, Sun L, Sun LY, Guo YM (2018) An automatic recognition method for PCB visual defects. In: 2018 International Conference on Sensing, Diagnostics, Prognostics, and Control (Sdpc), pp 138–142. https://doi.org/10.1109/Sdpc.2018.00034

    Chapter  Google Scholar 

  67. Kumar A (2003) Neural network based detection of local textile defects. Pattern Recogn 36(7):1645–1659. https://doi.org/10.1016/S0031-3203(03)00005-0

    Article  Google Scholar 

  68. Kang GW, Liu HB (2005) Surface defects inspection of cold rolled strips based on neural network. In: 2005 International Conference on Machine Learning and Cybernetics 8:5034–5037. https://doi.org/10.1109/ICMLC.2005.1527830

  69. Yang CH, Zhang JX, Ji G, Fu YJ, Hong X (2007) Recognition of defects in steel surface image based on neural networks and morphology. In: Second Workshop on Digital Media and Its Application in Museum & Heritage, Proceedings, pp 72–75. https://doi.org/10.1109/Dmamh.2007.56

    Chapter  Google Scholar 

  70. Ashour MW, Hussin MF, Mahar KM (2008) Supervised texture classification using several features extraction techniques based on ANN and SVM. I C Comput Syst Appl:567–574. https://doi.org/10.1109/Aiccsa.2008.4493588

  71. Chen LF, Su CT, Chen MH (2009) A neural-network approach for defect recognition in TFT-LCD photolithography process. IEEE Trans Electron Packag Manuf 32(1):1–8

    Article  Google Scholar 

  72. Tseng DC, Chung IL, Tsai PL, Chou CM (2011) Defect classification for Lcd color filters using neural-network decision tree classifier. Int J Innov Comput I 7(7a):3695–3707

    Google Scholar 

  73. Kwon BG, Kang DJ (2011) Fast defect detection algorithm on the variety surface with random forest using GPUs. In: 2011 11th International Conference on Control, Automation and Systems (Iccas), pp 1135–1136

  74. Tseng DC, Liu YS, Chou CM (2015) Automatic finger interruption detection in electroluminescence images of multicrystalline solar cells. Math Probl Eng 2015:1–12. https://doi.org/10.1155/2015/879675

    Article  Google Scholar 

  75. Hu H, Liu Y, Liu M, Nie L (2016) Surface defect classification in large-scale strip steel image collection via hybrid chromosome genetic algorithm. Neurocomputing 181:86–95

    Article  Google Scholar 

  76. Tian SY, Xu K (2017) An algorithm for surface defect identification of steel plates based on genetic algorithm and extreme learning machine. Metals-Basel 7(8). https://doi.org/10.3390/met7080311

  77. Piao M, Jin CH, Lee JY, Byun JY (2018) Decision tree ensemble-based wafer map failure pattern recognition based on radon transform-based features. IEEE Trans Semicond Manuf 31(2):250–257. https://doi.org/10.1109/Tsm.2018.2806931

    Article  Google Scholar 

  78. Celik HI, Dulger LU, Topalbekiroglu M (2014) Development of a machine vision system: real-time fabric defect detection and classification with neural networks. J Text Inst 105(6):575–585. https://doi.org/10.1080/00405000.2013.827393

    Article  Google Scholar 

  79. Wang CH, Wang SJ, Lee WD (2006) Automatic identification of spatial defect patterns for semiconductor manufacturing. Int J Prod Res 44(23):5169–5185. https://doi.org/10.1080/02772240600610822

    Article  MATH  Google Scholar 

  80. Nguyen VH, Pham VH, Cui X, Ma M, Kim H (2017) Design and evaluation of features and classifiers for OLED panel defect recognition in machine vision. J Inf Telecommun:334–350. https://doi.org/10.1080/24751839.2017.1355717

  81. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444. https://doi.org/10.1038/nature14539

    Article  Google Scholar 

  82. Bengio Y, Courville A, Vincent P (2013) Representation learning: a review and new perspectives. IEEE Trans Pattern Anal 35(8):1798–1828. https://doi.org/10.1109/Tpami.2013.50

    Article  Google Scholar 

  83. Deng J, Dong W, Socher R, Li LJ, Li K, Li FF (2009) ImageNet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp 248–255. https://doi.org/10.1109/CVPR.2009.5206848

  84. Feng X, Jiang Y, Yang X, Du M, Li X (2019) Computer vision algorithms and hardware implementations: a survey. Integration. 69:309–320. https://doi.org/10.1016/j.vlsi.2019.07.005

    Article  Google Scholar 

  85. Krizhevsky A, Sutskever I, Hinton GE (2012) ImageNet classification with deep convolutional neural networks. In: 2012 25th International Conference on Neural Information Processing Systems 1:1097–1105. https://doi.org/10.5555/2999134.2999257

  86. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556. https://arxiv.org/abs/1409.1556

  87. Szegedy C, Liu W, Jia YQ, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: 2015 Ieee Conference on Computer Vision and Pattern Recognition (Cvpr), pp 1–9. https://doi.org/10.1109/CVPR.2015.7298594

  88. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. In: 2016 Ieee Conference on Computer Vision and Pattern Recognition (Cvpr), pp 2818–2826. https://doi.org/10.1109/Cvpr.2016.308

    Chapter  Google Scholar 

  89. He KM, Zhang XY, Ren SQ, Sun J (2016) Deep residual learning for image recognition. In: 2016 Ieee Conference on Computer Vision and Pattern Recognition (Cvpr), pp 770–778. https://doi.org/10.1109/Cvpr.2016.90

    Chapter  Google Scholar 

  90. Huang G, Liu Z, van der Maaten L, Weinberger K (2017) Densely connected convolutional networks. In: Conference on Computer Vision and Pattern Recognition, 2017. https://doi.org/10.1109/CVPR.2017.243

    Chapter  Google Scholar 

  91. Iandola FN, Han S, Moskewicz MW, Ashraf K, Dally WJ, Keutzer K (2016) SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size. arXiv:1602.07360. https://arxiv.org/abs/1602.07360

  92. Howard AG, Zhu M, Bo C, Kalenichenko D, Wang W, Weyand T, Andreetto M, Adam H (2017) MobileNets: efficient convolutional neural networks for mobile vision applications. arXiv:1704.04861. https://arxiv.org/abs/1704.04861

  93. Sifre L (2014) Rigid-motion scattering for image classification. Ecole Polytechnique, Paris

    Google Scholar 

  94. Sandler M, Howard A, Zhu M, Zhmoginov A, Chen L-C (2018) MobileNetV2: inverted residuals and linear bottlenecks. In: 2018 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 4510–4520. https://doi.org/10.1109/CVPR.2018.00474

  95. Zhang X, Zhou XY, Lin MX, Sun R (2018) ShuffleNet: an extremely efficient convolutional neural network for mobile devices. In: 2018 Ieee/Cvf Conference on Computer Vision and Pattern Recognition (Cvpr), pp 6848–6856. https://doi.org/10.1109/Cvpr.2018.00716

    Chapter  Google Scholar 

  96. Ma N, Zhang X, Zheng H-T, Jian S (2018) ShuffleNet V2: practical guidelines for efficient CNN architecture design. In: 2018 European Conference on Computer Vision (ECCV). https://doi.org/10.1007/978-3-030-01264-9_8

  97. Ren SQ, He KM, Girshick R, Sun J (2017) Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence 39(6):1137–1149. https://doi.org/10.1109/TPAMI.2016.2577031

  98. Girshick R (2015) Fast R-CNN. In: 2015 IEEE International Conference on Computer Vision (ICCV), pp 1440–1448. https://doi.org/10.1109/ICCV.2015.169

  99. Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: unified, real-time object detection. In: 2016 Ieee Conference on Computer Vision and Pattern Recognition (Cvpr), pp 779–788. https://doi.org/10.1109/Cvpr.2016.91

    Chapter  Google Scholar 

  100. Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu CY, Berg AC (2016) SSD: Single Shot MultiBox Detector. Computer vision - Eccv 2016, Pt I 9905:21–37. https://doi.org/10.1007/978-3-319-46448-0_2

  101. Rumelhart DE (1986) Learning representations by back-propagating errors. Nature. https://doi.org/10.1016/B978-1-4832-1446-7.50035-2

  102. Hinton GE, Salakhutdinov RR (2006) Reducing the dimensionality of data with neural networks. Science 313(5786):504–507. https://doi.org/10.1126/science.1127647

    Article  MathSciNet  MATH  Google Scholar 

  103. Vincent P, Larochelle H, Bengio Y, Manzagol PA (2008) Extracting and composing robust features with denoising autoencoders. In: the 25th International Conference on Machine Learning (ICML 2008), pp 1096–1103. https://doi.org/10.1145/1390156.1390294

  104. Goodfellow IJ, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. Adv Neural Inf Proces Syst 27(nips 2014):27

    Google Scholar 

  105. Schlegl T, Seebock P, Waldstein SM, Langs G, Schmidt-Erfurth U (2019) f-AnoGAN: fast unsupervised anomaly detection with generative adversarial networks. Med Image Anal 54:30–44. https://doi.org/10.1016/j.media.2019.01.010

    Article  Google Scholar 

  106. Akcay S, Atapour-Abarghouei A, Breckon TP (2019) GANomaly: semi-supervised anomaly detection via adversarial training. Computer vision - Accv 2018, Pt Iii 11363:622–637. https://doi.org/10.1007/978-3-030-20893-6_39

  107. DAGM texture dataset. https://hci.iwr.uni-heidelberg.de/node/3616. Accessed Oct. 2019

  108. Wu MJ, Jang JSR, Chen JL (2015) Wafer map failure pattern recognition and similarity ranking for large-scale data sets. IEEE Trans Semicond Manuf 28(1):1–12. https://doi.org/10.1109/Tsm.2014.2364237

    Article  Google Scholar 

  109. Song KC, Yan YH (2013) A noise robust method based on completed local binary patterns for hot-rolled steel strip surface defects. Appl Surf Sci 285:858–864. https://doi.org/10.1016/j.apsusc.2013.09.002

    Article  Google Scholar 

  110. Tang S, He F, Huang X, Yang J (2019) Online PCB defect detector on a new PCB defect dataset

  111. Huang YB, Qiu CY, Guo Y, Wang XN, Yuan K (2018) Surface defect saliency of magnetic tile. Ieee Int Con Auto Sc:612–617

  112. Deitsch S, Christlein V, Berger S, Buerhop-Lutz C, Maier A, Gallwitz F, Riess C (2019) Automatic classification of defective photovoltaic module cells in electroluminescence images. Sol Energy 185:455–468

    Article  Google Scholar 

  113. Gan JR, Li QT, Wang JZ, Yu HM (2017) A hierarchical extractor-based visual rail surface inspection system. IEEE Sensors J 17(23):7935–7944. https://doi.org/10.1109/Jsen.2017.2761858

    Article  Google Scholar 

  114. TILDA Textile Texture-Database (1996). https://lmb.informatik.uni-freiburg.de/resources/datasets/tilda.en.html. Accessed Oct. 2019 2019

  115. Kylberg G (2011) The Kylberg Texture Dataset v. 1.0. External report (Blue series) vol No. 35. Centre for Image Analysis, Swedish University of Agricultural Sciences and Uppsala University. http://www.cb.uu.se/~gustaf/texture/. Accessed 19 Jan 2021

  116. Kampouris C, Zafeiriou S, Ghosh A, Malassiotis S (2016) Fine-grained material classification using micro-geometry and reflectance. Computer vision - Eccv 2016, Pt V 9909:778–792. https://doi.org/10.1007/978-3-319-46454-1_47

  117. Fritz M, Hayman E, Caputo B, Eklundh J-O (2019) The KTH-TIPS database. Accessed Oct. 2019

  118. Li YY, Zhang D, Lee DJ (2019) Automatic fabric defect detection with a wide-and-compact network. Neurocomputing 329:329–338. https://doi.org/10.1016/j.neucom.2018.10.070

    Article  Google Scholar 

  119. Michalski P, Ruszczak B, Tomaszewski M (2018) Convolutional neural networks implementations for computer vision. Adv Intell Syst 720:98–110. https://doi.org/10.1007/978-3-319-75025-5_10

    Article  Google Scholar 

  120. Caggiano A, Zhang JJ, Alfieri V, Caiazzo F, Gao R, Teti R (2019) Machine learning-based image processing for on-line defect recognition in additive manufacturing. Cirp Ann Manuf Technol 68(1):451–454. https://doi.org/10.1016/j.cirp.2019.03.021

    Article  Google Scholar 

  121. Yang H, Mei S, Song K, Tao B, Yin Z (2018) Transfer-learning-based online Mura defect classification. IEEE Trans Semicond Manuf 31(1):116–123. https://doi.org/10.1109/TSM.2017.2777499

    Article  Google Scholar 

  122. Kim Y-G, Lim D-U, Ryu J-H, Park T-H SMD Defect classification by convolution neural network and PCB image transform. In: 3rd IEEE International Conference on Computing, Communication and Security, ICCCS 2018, October 25, 2018 - October 27, 2018, Kathmandu, Nepal, 2018. Proceedings on 2018 IEEE 3rd International Conference on Computing, Communication and Security, ICCCS 2018. Institute of Electrical and Electronics Engineers Inc, pp 180–183. https://doi.org/10.1109/CCCS.2018.8586818

  123. Kim J, Kim S, Kwon N, Kang H, Kim Y, Lee C Deep learning based automatic defect classification in through-silicon Via process: FA: Factory automation. In: 29th Annual SEMI Advanced Semiconductor Manufacturing Conference, Saratoga Springs, NY, United states, 2018 2018. 2018 29th Annual SEMI Advanced Semiconductor Manufacturing Conference, ASMC 2018. Institute of Electrical and Electronics Engineers Inc, pp 35–39. https://doi.org/10.1109/ASMC.2018.8373144

  124. Jang C, Yun S, Hwang H, Shin H, Kim SS, Park Y (2018) A defect inspection method for machine vision using defect probability image with deep convolutional neural network. In: 2018 Asian Conference on Computer Vision (ACCV ), pp 142–154. https://doi.org/10.1007/978-3-030-20887-5_9

  125. Zhang L, Jin Y, Yang X, Li X, Duan X, Sun Y, Liu H (2018) Convolutional neural network-based multi-label classification of PCB defects. J Eng 16:1612–1616. https://doi.org/10.1049/joe.2018.8279

    Article  Google Scholar 

  126. Deng Y-S, Luo A-C, Dai M-J Building an automatic defect verification system using deep neural network for PCB defect classification. In: 4th International Conference on Frontiers of Signal Processing, ICFSP 2018, September 24, 2018 - September 27, 2018, Poitiers, France, 2018. 2018 4th International Conference on Frontiers of Signal Processing, ICFSP 2018. Institute of Electrical and Electronics Engineers Inc, pp 145–149. https://doi.org/10.1109/ICFSP.2018.8552045

  127. Ghosh B, Bhuyan MK, Sasmal P, Iwahori Y, Gadde P Defect classification of printed circuit boards based on transfer learning. In: 2018 IEEE Applied Signal Processing Conference, ASPCON 2018, December 7, 2018 - December 9, 2018, Kolkata, India, 2018. Proceedings of 2018 IEEE Applied Signal Processing Conference, ASPCON 2018. Institute of Electrical and Electronics Engineers Inc, pp 245–248. https://doi.org/10.1109/ASPCON.2018.8748670

  128. Wei P, Liu C, Liu M, Gao Y, Liu H (2018) CNN based reference comparison method for classifying bare PCB defects. J Eng 2018(16):1528–1533. https://doi.org/10.1049/joe.2018.8271

    Article  Google Scholar 

  129. Nakazawa T, Kulkarni DV (2018) Wafer map defect pattern classification and image retrieval using convolutional neural network. IEEE Trans Semicond Manuf 31(2):309–314. https://doi.org/10.1109/TSM.2018.2795466

    Article  Google Scholar 

  130. Yuan-Fu Y (2019) A deep learning model for identification of defect patterns in semiconductor wafer map. In: 30th Annual SEMI Advanced Semiconductor Manufacturing Conference, ASMC 2019. Institute of Electrical and Electronics Engineers Inc. https://doi.org/10.1109/ASMC.2019.8791815

  131. Ishida T, Nitta I, Fukuda D, Kanazawa Y (2019) Deep learning-based wafer-map failure pattern recognition framework. In: 2019 20th International Symposium on Quality Electronic Design (Isqed), pp 291–297. https://doi.org/10.1109/ISQED.2019.8697407

  132. Cheon S, Lee H, Kim CO, Lee SH (2019) Convolutional neural network for wafer surface defect classification and the detection of unknown defect class. IEEE Trans Semicond Manuf 32(2):163–170. https://doi.org/10.1109/Tsm.2019.2902657

    Article  Google Scholar 

  133. Banda P, Barnard L A deep learning approach to photovoltaic cell defect classification. In: 2018 Annual Conference of the South African Institute of Computer Scientists and Information Technologists: Technology for Change, Port Elizabeth, South Africa, 2018 2018. ACM International Conference Proceeding Series. Association for Computing Machinery, pp 215–221. https://doi.org/10.1145/3278681.3278707

  134. Lin H, Li B, Wang XG, Shu YF, Niu SL (2019) Automated defect inspection of LED chip using deep convolutional neural network. J Intell Manuf 30(6):2525–2534. https://doi.org/10.1007/s10845-018-1415-x

    Article  Google Scholar 

  135. Park JK, Kwon BK, Park JH, Kang DJ (2016) Machine learning-based imaging system for surface defect inspection. Int J Precis Eng Manuf Green Technol 3(3):303–310

    Article  Google Scholar 

  136. Weimer D, Scholz-Reiter B, Shpitalni M (2016) Design of deep convolutional neural network architectures for automated feature extraction in industrial inspection. Cirp Ann Manuf Technol 65(1):417–420. https://doi.org/10.1016/j.cirp.2016.04.072

    Article  Google Scholar 

  137. Wang T, Chen Y, Qiao MN, Snoussi H (2018) A fast and robust convolutional neural network-based defect detection model in product quality control. Int J Adv Manuf Technol 94(9–12):3465–3471. https://doi.org/10.1007/s00170-017-0882-0

    Article  Google Scholar 

  138. Jeyaraj PR, Samuel Nadar ER (2019) Computer vision for automatic detection and classification of fabric defect employing deep learning algorithm. Int J Cloth Sci Technol 31(4):510–521. https://doi.org/10.1108/IJCST-11-2018-0135

    Article  Google Scholar 

  139. Saiz FA, Serrano I, Barandiaran I, Sanchez JR A robust and fast deep learning-based method for defect classification in steel surfaces. In: 9th International Conference on Intelligent Systems, IS 2018, September 25, 2018 - September 27, 2018, Funchal - Madeira, Portugal, 2018. 9th International Conference on Intelligent Systems 2018: Theory, Research and Innovation in Applications, IS 2018 - Proceedings. Institute of Electrical and Electronics Engineers Inc, pp 455–460. https://doi.org/10.1109/IS.2018.8710501

  140. Chen W, Gao Y, Gao L, Li XA (2018) New ensemble approach based on deep convolutional neural networks for steel surface defect classification. In: 51st CIRP Conference on Manufacturing Systems, CIRP CMS 2018, May 16, 2018 - May 18, 2018, Stockholm, Sweden. Elsevier B.V, pp 1069–1072. https://doi.org/10.1016/j.procir.2018.03.264

  141. Liu Z, Wang X, Chen X Inception dual network for steel strip defect detection. In: 16th IEEE International Conference on Networking, Sensing and Control, ICNSC 2019, May 9, 2019 - May 11, 2019, Banff, AB, Canada, 2019. Proceedings of the 2019 IEEE 16th International Conference on Networking, Sensing and Control, ICNSC 2019. Institute of Electrical and Electronics Engineers Inc, pp 409–414. https://doi.org/10.1109/ICNSC.2019.8743190

  142. Vannocci M, Ritacco A, Castellano A, Galli F, Vannucci M, Iannino V, Colla V Flatness defect detection and classification in hot rolled steel strips using convolutional neural networks. In: 15th International Work-Conference on Artificial Neural Networks, IWANN 2019, June 12, 2019 - June 14, 2019, Gran Canaria, Spain, 2019. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), pp 220–234. https://doi.org/10.1007/978-3-030-20518-8_19

  143. Song LM, Li XY, Yang YG, Zhu XJ, Guo QH, Yang HD (2018) Detection of micro-defects on metal screw surfaces based on deep convolutional neural networks. Sensors-Basel 18(11). https://doi.org/10.3390/s18113709

  144. Chun LP, Zhao QF (2018) Product surface defect detection based on deep learning. In: 2018 16th Ieee Int Conf on Dependable, Autonom and Secure Comp, pp 250–255. https://doi.org/10.1109/DASC/PiCom/DataCom/CyberSciTec.2018.00051

    Chapter  Google Scholar 

  145. Soukup D, Huber-Mork R (2014) Convolutional neural networks for steel surface defect detection from photometric stereo images. Advances in visual computing (Isvc 2014), Pt 1 8887:668–677

  146. Mei S, Wang YD, Wen GJ (2018) Automatic fabric defect detection with a multi-scale convolutional denoising autoencoder network model. Sensors-Basel 18(4). https://doi.org/10.3390/s18041064

  147. Mujeeb A, Dai WT, Erdt M, Sourin A (2018) Unsupervised surface defect detection using deep autoencoders and data augmentation. In: 2018 International Conference on Cyberworlds (Cw), pp 391–398. https://doi.org/10.1109/Cw.2018.00076

    Chapter  Google Scholar 

  148. Siegmund D, Prajapati A, Kirchbuchner F, Kuijper A (2018) An integrated deep neural network for defect detection in dynamic textile textures. In: Progress in Artificial Intelligence and Pattern Recognition, Iwaipr 2018, vol 11047, pp 77–84. https://doi.org/10.1007/978-3-030-01132-1_9

    Chapter  Google Scholar 

  149. Li JY, Su ZF, Geng JH, Yin YX (2018) Real-time detection of steel strip surface defects based on improved YOLO detection network. IFAC-PapersOnLine 51(21):76–81. https://doi.org/10.1016/j.ifacol.2018.09.412

    Article  Google Scholar 

  150. Li YT, Huang HS, Xie QS, Yao LG, Chen QP (2018) Research on a surface defect detection algorithm based on MobileNet-SSD. Appl Sci-Basel 8(9). https://doi.org/10.3390/app8091678

  151. Yang J, Li S, Wang Z, Yang G (2019) Real-time tiny part defect detection system in manufacturing using deep learning. IEEE Access 7:89278–89291. https://doi.org/10.1109/ACCESS.2019.2925561

    Article  Google Scholar 

  152. Di H, Ke X, Peng Z, Zhou D (2019) Surface defect classification of steels with a new semi-supervised learning method. Opt Lasers Eng 117(1):40–48

    Article  Google Scholar 

  153. Gao YP, Gao L, Li XY, Yan XG (2020) A semi-supervised convolutional neural network-based method for steel surface defect recognition. Robot Comput Integr Manuf 61:101825. https://doi.org/10.1016/j.rcim.2019.101825

    Article  Google Scholar 

  154. Tan CQ, Sun FC, Kong T, Zhang WC, Yang C, Liu CF (2018) A survey on deep transfer learning. Artificial neural networks and machine learning - Icann 2018, Pt Iii 11141:270–279. https://doi.org/10.1007/978-3-030-01424-7_27

  155. Liu SP, Tian GH, Xu Y (2019) A novel scene classification model combining ResNet based transfer learning and data augmentation with a filter. Neurocomputing 338:191–206. https://doi.org/10.1016/j.neucom.2019.01.090

    Article  Google Scholar 

  156. Zheng X, Chen J, Wang H, Zheng S, Kong Y (2020) A deep learning-based approach for the automated surface inspection of copper clad laminate images. Applied Intelligence. https://doi.org/10.1007/s10489-020-01877-z

  157. Zhu XJ (2005) Semi-supervised learning literature survey. University of Wisconsin-Madison Department of Computer Sciences. http://digital.library.wisc.edu/1793/60444. Accessed 19 Jan 2021

  158. Chapelle O, Schölkopf B, Zien A (2006) Semi-supervised learning, vol 2. MIT Press Cortes, Cambridge

    Book  Google Scholar 

  159. Cortes C, Mohri M (2014) Domain adaptation and sample bias correction theory and algorithm for regression. Theor Comput Sci 519:103126

    Article  MathSciNet  MATH  Google Scholar 

  160. Odena A (2016) Semi-supervised learning with generative adversarial networks. arXiv:1606.01583. https://arxiv.org/abs/1606.01583

  161. Li W, Wang Z, Li J, Polson J, Speier W, Arnold CW (2019) Semi-supervised learning based on generative adversarial network: a comparison between good GAN and bad GAN approach. In: 2019 Computer Vision and Pattern Recognition (CVPR) Workshops. arXiv:1905.06484. https://arxiv.org/abs/1905.06484

  162. Berthelot D, Carlini N, Goodfellow I, Papernot N, Oliver A, Raffel CA (2019) Mixmatch: a holistic approach to semi-supervised learning. In: Advances in Neural Information Processing Systems, pp 5049–5059

    Google Scholar 

  163. Zheng X, Wang H, Chen J, Kong Y, Zheng S (2020) A generic semi-supervised deep learning-based approach for automated surface inspection. IEEE Access 8:114088–114099

    Article  Google Scholar 

Download references

Funding

This work was supported in part by the National Natural Science Foundation of China under grant number U1609212, Zhejiang Provincial Science and Technology Plan under grant number 2019C04021, and Zhejiang Province Public Technology Research Project under grant number LGG20F030002.

Author information

Authors and Affiliations

  1. School of Automation, Hangzhou Dianzi University, Hangzhou, 310018, China

    Xiaoqing Zheng, Song Zheng, Yaguang Kong & Jie Chen

Authors
  1. Xiaoqing Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Song Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yaguang Kong
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jie Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaoqing Zheng.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

Not applicable.

Consent to participate

Not applicable.

Consent to publish

Not applicable.

Code availability

Not applicable.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zheng, X., Zheng, S., Kong, Y. et al. Recent advances in surface defect inspection of industrial products using deep learning techniques. Int J Adv Manuf Technol 113, 35–58 (2021). https://doi.org/10.1007/s00170-021-06592-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00170-021-06592-8

Keywords

Search

Navigation