Skip to main content
SpringerLink
Log in

A real-time defective pixel detection system for LCDs using deep learning based object detectors

  • Published:
Journal of Intelligent Manufacturing Aims and scope Submit manuscript

Abstract

The presence of pixel defects on the screens of LCD-based products (TV, tablet, phone, etc.) is unacceptable given the consumer expectations. Therefore, these defects should be detected before the product reaches the user during the production stage. Visual inspections are mostly performed by human operators in the production. These inspections are error prone and not efficient in terms of consumed time. For this reason, computer visionbased approaches are started to find applications in this kind of problems. This paper presents an image acquisition system and a detailed analysis of deep learningbased object detectors for LCD pixel defect detection problem. Experimental results show that the proposed methods can be a powerful alternative to operator control by providing more efficient use of time, human, financial resources and betterquality standards in TV production industry.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  • Bi, X., Xu, X., & Shen, J. (2015). An automatic detection method of Mura defects for liquid crystal display using real Gabor filters. In 2015 8th International Congress on Image and Signal Processing (CISP), (pp. 871–875). IEEE.

  • Chen, L. F., Su, C. T., & Chen, M. H. (2009). A neural-network approach for defect recognition in TFT-LCD photolithography process. IEEE Transactions on Electronics Packaging Manufacturing, 32(1), 1–8.

    Article  Google Scholar 

  • Chen, T., Chen, -L., Y., -Y., & Lu, H., -C (2013). A capacity allocation and expansion model for TFT-LCD multi-site manufacturing. Journal of Intelligent Manufacturing, 24, 857–872.

    Article  Google Scholar 

  • Çelik, A., Sümer, A., Küçükmanisa, A., Taşyapı Çelebi, A., & Urhan, O. (2018). LCD Pixel defect detection using GLCM features and SVM. International Marmara Science and Social Sciences Congress, (pp 853–858).

  • Çelik, A., Küçükmanisa, A., Sümer, A., Taşyapı Çelebi, A., & Urhan, O. (2020). LCD pixel defect detection using a shallow CNN based approach. Electrica, In Review.

  • Fan, S. K. S., & Chuang, Y. C. (2010). Automatic detection of Mura defect in TFT-LCD based on regression diagnostics. Pattern Recognition Letters, 31(15), 2397–2404.

    Article  Google Scholar 

  • Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014). Rich feature hierarchies for accurate object detection and semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, (pp. 580–587).

  • Girshick, R. (2015). Fast r-cnn. In Proceedings of the IEEE international conference on computer vision, (pp. 1440–1448).

  • Guo, B., Hu, G. Q., & Yang, G. Y. (2014). TFT-LCD spot-type defect detection in module process. InAdvanced Materials Research, 971–973, 1368–1371.

    Article  Google Scholar 

  • Guo, L., Li, S., Hu, W., Wu, J., Tu, B., He, W., Ou, X., & Zhang, G. (2018). Sub-pixel level defect detection based on notch filter and image registration. International Journal of Pattern Recognition and Artificial Intelligence, 32(06), 1854016.

    Article  Google Scholar 

  • He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (pp. 770–778).

  • Jin, S., Ji, C., Yan, C., & Xing, J. (2018). TFT-LCD mura defect detection using DCT and the dual-γ piecewise exponential transform. Precision Engineering, 54, 371–378.

    Article  Google Scholar 

  • Juen, K., Ki, B. L., Jaeyeon, J., Kyong, S. C., & Chang, O. K. (2019). Automatic inspection of salt-and-pepper defects in OLED panels using image processing and control chart techniques. Journal of Intelligent Manufacturing, 30(1), 1047–1055.

    Google Scholar 

  • Kim, M., Lee, M., An, M., & Lee, H. (2020). Effective automatic defect classification process based on CNN with stacking ensemble model for tft-lcd panel. Journal of Intelligent Manufacturing, 31, 1165–1174.

    Article  Google Scholar 

  • Krizhevsky, A., Sutskever, I., & Hinton, G. (2012). Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems, 1097–1105.

  • LeCun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278–2324.

  • Li, D.-C., Chen, W.-C., & Liu, C.-W. (2012). A non-linear quality improvement model using SVR for manufacturing TFT-LCDs. Journal of Intelligent Manufacturing, 23, 835–844.

    Article  Google Scholar 

  • Lin, C. S., Liao, Y. C., Lay, Y. L., Lee, K. C., & Yeh, M. S. (2008). High-speed TFT LCD defect‐detection system with genetic algorithm. Assembly Automation, 28(1), 69–76.

    Article  Google Scholar 

  • Lin, T. Y., Goyal, P., Girshick, R., He, K., & Dollár, P. (2017). Focal loss for dense object detection. In Proceedings of the IEEE Iiternational conference on computer vision, (pp. 2980–2988).

  • Liu, Y. H., Huang, Y. K., & Lee, M. J. (2008). Automatic inline defect detection for a thin film transistor–liquid crystal display array process using locally linear embedding and support vector data description. Measurement Science and Technology, 19(9), 095501.

    Article  Google Scholar 

  • Liu, Y. H., Liu, Y. C., & Chen, Y. Z. (2011). High-speed inline defect detection for TFT-LCD array process using a novel support vector data description. Expert Systems with Applications, 38(5), 6222–6231.

    Article  Google Scholar 

  • Mei, S., Yang, H., & Yin, Z. (2017). Unsupervised-learning-based feature-level fusion method for mura defect recognition. IEEE Transactions on Semiconductor Manufacturing, 30(1), 105–113.

    Article  Google Scholar 

  • Otsu, N. (1979). A threshold selection method from gray-level histograms. IEEE Transactions on Systems, Man, and Cybernetics, 9(1), 62–66.

    Article  Google Scholar 

  • Qian, J., Li, W., Chen, B., Qian, J., Liu, F., & Zhou, X. (2018). Sub-Pixel defect detection for super large LCD images using quadrant gradient operator. In 2018 IEEE 18th International Conference on Communication Technology (ICCT) (pp. 1183–1187). IEEE.

  • Ramya, B. S., Gaurav, K., Gaurav, S., Shashank, S., Priya, R.. Chulmoo, K. Deep learning based MURA defect detection. EAI Endorsed Transactions on Cloud Systems, 5(15), 1–7.

    Google Scholar 

  • Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016). You only look once: Unified, real-time object detection. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, (pp. 779–788).

  • Redmon, J., & Farhadi, A. (2018). Yolov3: An incremental improvement. arXiv preprint arXiv:1804.02767.

  • Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556.

  • Song, Y. C., Choi, D. H., & Park, K. H. (2004). Morphological blob-Mura defect detection method for TFT-LCD panel inspection. InInternational Conference on Knowledge-Based and Intelligent Information and Engineering Systems (pp. 862–868).

  • Su, X. H., He, Z. G., & Ma, P. J. (2008). An automatic detection algorithm for TFT-LCD micro display defects. Journal of Harbin Institute of Technology, 40(11), 1756–1760.

    Google Scholar 

  • Sümer, A., Çelik, A., Küçükmanisa, A., Çelebi, A. T., & Urhan, O. (2019). Pixel Defect Detection in LCD TV Images using Adaptive Thresholding. In 2019 27th signal processing and communications applications conference (SIU), (pp. 1–4). IEEE.

  • Wang, X., Dong, R., & Li, B. (2016). TFT-LCD mura defect detection based on ICA and multi-channels fusion. In 2016 3rd International conference on information science and control engineering (ICISCE), (pp. 687–691). IEEE.

  • Yang, S. W., Lin, C. S., Lin, S. K., & Chiang, H. T. (2014). Automatic defect recognition of TFT array process using gray level co-occurrence matrix. Optik-International Journal for Light and Electron Optics, 125(11), 2671–2676.

    Article  Google Scholar 

  • Yang, H., Mei, S., Song, K., Tao, B., & Yin, Z. (2018). Transfer-learning-based online Mura defect classification. IEEE Transactions on Semiconductor Manufacturing, 31(1), 116–123.

    Article  Google Scholar 

  • Zhang, Y., & Zhang, J. (2005). A fuzzy neural network approach for quantitative evaluation of mura in TFT-LCD. In 2005 International Conference on Neural Networks and Brain (pp. 424–427). IEEE.

  • Zhao, Q., Sheng, T., Wang, Y., Tang, Z., Chen, Y., Cai, L., & Ling, H. (2019). M2det: A single-shot object detector based on multi-level feature pyramid network. InProceedings of the AAAI conference on artificial intelligence, (pp. 9259–9266).

Download references

Acknowledgements

This work is supported by the Scientific and Technological Research Council of Turkey (TUBITAK) under Grant No. 5150099 and Nvidia under Grant No. GPU Grant.

Author information

Authors and Affiliations

  1. Faculty of Engineering, Electronics and Communication Engineering, Kocaeli University, Kocaeli, Turkey

    Aslı Çelik, Ayhan Küçükmanisa, Aydın Sümer, Aysun Taşyapı Çelebi & Oğuzhan Urhan

Authors
  1. Aslı Çelik
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ayhan Küçükmanisa
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Aydın Sümer
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Aysun Taşyapı Çelebi
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Oğuzhan Urhan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ayhan Küçükmanisa.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Çelik, A., Küçükmanisa, A., Sümer, A. et al. A real-time defective pixel detection system for LCDs using deep learning based object detectors. J Intell Manuf 33, 985–994 (2022). https://doi.org/10.1007/s10845-020-01704-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10845-020-01704-9

Keywords

Search

Navigation