A Deep Learning-Based Surface Defect Inspection System for Smartphone Glass

  • Gwang-Myong Go
  • Seok-Jun Bu
  • Sung-Bae ChoEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11871)


In recent years, convolutional neural network has become a solution to many image processing problems due to high performance. It is particularly useful for applications in automated optical inspection systems related to industrial applications. This paper proposes a system that combines the defect information, which is meta data, with the defect image by modeling. Our model for classification consists of a separate model for embedding location information in order to utilize the defective locations classified as defective candidates and ensemble with the model for classification to enhance the overall system performance. The proposed system incorporates class activation map for preprocessing and augmentation for image acquisition and classification through optical system, and feedback of classification performance by constructing a system for defect detection. Experiment with real-world dataset shows that the proposed system achieved 97.4% accuracy and through various other experiments, we verified that our system is applicable.


Deep learning Convolutional neural network Class activation map Smartphone glass inspection Defect detection Augmentation Image preprocessing 



This research was supported by Samsung Electronics Co., Ltd.


  1. 1.
    Sun, J., Wang, P., Luo, Y.K., Li, W.: Surface defects detection based on adaptive multiscale image collection and convolutional neural networks. IEEE Trans. Instrum. Measur. 1–11 (2019)Google Scholar
  2. 2.
    Natarajan, V., Hung, T.Y., Vaikundam, S., Chia, L.T.: Convolutional networks for voting-based anomaly classification in metal surface inspection. In: IEEE International Conference on Industrial Technology, pp. 986–991 (2017)Google Scholar
  3. 3.
    Chen, T., Wang, Y., Xiao, C., Wu, Q.J.: A machine vision apparatus and method for can-end inspection. IEEE Trans. Instrum. Measur. 65, 2055–2066 (2016)CrossRefGoogle Scholar
  4. 4.
    Cao, G., Ruan, S., Peng, Y., Huang, S., Kwok, N.: Large-complex-surface defect detection by hybrid gradient threshold segmentation and image registration. IEEE Access 6, 36235–36246 (2018)CrossRefGoogle Scholar
  5. 5.
    Jian, C., Gao, J., Ao, Y.: Automatic surface defect detection for mobile phone screen glass based on machine vision. Appl. Soft Comput. 52, 348–358 (2017)CrossRefGoogle Scholar
  6. 6.
    Stark, J.A.: Adaptive image contrast enhancement using generalizations of histogram equalization. IEEE Trans. Image Process. 9, 889–896 (2000)CrossRefGoogle Scholar
  7. 7.
    Gupta, E., Kushwah, R.S.: Combination of global and local features using DWT with SVM for CBIR. In: International Conference on Reliability, Infocom Technologies and Optimization, pp. 1–6 (2015)Google Scholar
  8. 8.
    Zhou, B., Khosla, A., Lapedriza, A., Olivia, A., Torralba, A.: Learning deep features for discriminative localization. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2921–2929 (2016)Google Scholar
  9. 9.
    Borwankar, R., Ludwig, R.: An optical surface inspection and automatic classification technique using the rotated wavelet transform. IEEE Trans. Instrum. Measur. 67, 690–697 (2018)CrossRefGoogle Scholar
  10. 10.
    Dong, X., Taylor, C.J., Cootes, T.F.: Small defect detection using convolutional neural network features and random forests. In: European Conference on Computer Vision, pp. 1–15 (2018)Google Scholar
  11. 11.
    Fu, G., et al.: A deep-learning-based approach for fast and robust steel surface defects classification. Opt. Lasers Eng. 121, 397–405 (2019)CrossRefGoogle Scholar
  12. 12.
    Yang, H., Chen, Y., Song, K., Yin, Z.: Multiscale feature-clustering-based fully convolutional autoencoder for fast accurate visual inspection of texture surface defects. IEEE Trans. Autom. Sci. Eng. 99, 1–18 (2019)Google Scholar
  13. 13.
    Staar, B., Lutjen, M., Freitag, M.: Anomaly detection with convolutional neural networks for industrial surface inspection. Procedia CIRP. 79, 484–489 (2019)CrossRefGoogle Scholar
  14. 14.
    Sainath, T., Parada, C.: Convolutional neural networks for small-footprint keyword spotting. In: InterSpeech, pp. 1478–1482 (2015)Google Scholar
  15. 15.
    Donahue, J., et al.: Long-term recurrent convolutional networks for visual recognition and description. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2625–2634 (2015)Google Scholar
  16. 16.
    Rippel, O., Snoek, J., Adams, R.P.: Spectral representations for convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 2449–2457 (2015)Google Scholar
  17. 17.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Department of Computer ScienceYonsei UniversitySeoulSouth Korea

Personalised recommendations