Skip to main content
SpringerLink
Log in

Multiple Detection Model Fusion Framework for Printed Circuit Board Defect Detection

基于多重检测模型融合框架的印刷电路板缺陷检测

  • Published:
Journal of Shanghai Jiaotong University (Science) Aims and scope Submit manuscript

Abstract

The printed circuit board (PCB) is an indispensable component of electronic products, which determines the quality of these products. With the development and advancement of manufacturing technology, the layout and structure of PCB are getting complicated. However, there are few effective and accurate PCB defect detection methods. There are high requirements for the accuracy of PCB defect detection in the actual production environment, so we propose two PCB defect detection frameworks with multiple model fusion including the defect detection by multi-model voting method (DDMV) and the defect detection by multi-model learning method (DDML). With the purpose of reducing wrong and missing detection, the DDMV and DDML integrate multiple defect detection networks with different fusion strategies. The effectiveness and accuracy of the proposed framework are verified with extensive experiments on two open-source PCB datasets. The experimental results demonstrate that the proposed DDMV and DDML are better than any other individual state-of-the-art PCB defect detection model in F1-score, and the area under curve value of DDML is also higher than that of any other individual detection model. Furthermore, compared with DDMV, the DDML with an automatic machine learning method achieves the best performance in PCB defect detection, and the F1-score on the two datasets can reach 99.7% and 95.6% respectively.

摘要

印刷电路板(PCB)是电子产品不可或缺的组成部分,它决定了这些产品的质量。随着制造技术的发展和进步,PCB的布局和结构变得越来越复杂。然而,有效准确的PCB缺陷检测方法却很少。在实际生产环境中,对PCB缺陷检测的准确性有很高的要求,因此我们提出了两种包括多模型融合的PCB缺陷检测框架,包括多模型投票方法(DDMV)和多模型学习方法(DDML)的缺陷检测。为了减少错误和漏检,DDMV和DDML将多个具有不同融合策略的缺陷检测网络进行整合。通过对两个开源PCB数据集进行大量实验,验证了所提出框架的有效性和准确性。实验结果表明,所提出的DDMV和DDML在F1分数方面优于其他任何单独的最先进的PCB缺陷检测模型,而DDML的曲线下面积值也高于其他任何单独的检测模型。此外,与DDMV相比,使用自动机器学习方法的DDML在PCB缺陷检测方面达到了最好的性能,在两个数据集上的F1分数分别可达到99.7%和95.6%。

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. CHAUDHARY V, DAVE I R, UPLA K P. Automatic visual inspection of printed circuit board for defect detection and classification [C]//2017 International Conference on Wireless Communications, Signal Processing and Networking. Chennai: IEEE, 2017: 732–737.

    Google Scholar 

  2. ZHU JH, WU A, LIU X P. Printed circuit board defect visual detection based on wavelet denoising [J]. IOP Conference Series: Materials Science and Engineering, 2018, 392: 062055.

    Article  Google Scholar 

  3. KUO C F J, FANG T Y, LEE C L, et al. Automated optical inspection system for surface mount device light emitting diodes [J]. Journal of Intelligent Manufacturing, 2019, 30(2): 641–655.

    Article  Google Scholar 

  4. KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks [J]. Communications of the ACM, 2017, 60(6): 84–90.

    Article  Google Scholar 

  5. WU X, ZHONG M Y, GUO Y K, et al. The assessment of small bowel motility with attentive deformable neural network [J]. Information Sciences, 2020, 508: 22–32.

    Article  Google Scholar 

  6. LIU W, ANGUELOV D, ERHAN D, et al. SSD: Single shot multibox detector [M]//Computer vision -ECCV 2016. Cham: Springer, 2016: 21–37.

    Chapter  Google Scholar 

  7. REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: Unified, real-time object detection [C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV: IEEE, 2016: 779–788.

    Google Scholar 

  8. LIN T Y, DOLLÁR P, GIRSHICK R, et al. Feature pyramid networks for object detection [C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI: IEEE, 2017: 936–944.

    Google Scholar 

  9. JIANG H Z, LEARNED-MILLER E. Face detection with the faster R-CNN [C]//2017 12th IEEE International Conference on Automatic Face & Gesture Recognition. Washington, DC, USA: IEEE, 2017: 650–657.

    Google Scholar 

  10. WANG Y, LUO X B, DING L, et al. Detection based visual tracking with convolutional neural network [J]. Knowledge-Based Systems, 2019, 175: 62–71.

    Article  Google Scholar 

  11. WEI H, YANG C Z, YU Q. Efficient graph-based search for object detection [J]. Information Sciences, 2017, 385/386: 395–414.

    Article  Google Scholar 

  12. BRIA A, MARROCCO C, MOLINARA M, et al. An effective learning strategy for cascaded object detection [J]. Information Sciences, 2016, 340/341: 17–26.

    Article  MathSciNet  Google Scholar 

  13. OLSON R S, MOORE J H. TPOT: A tree-based pipeline optimization tool for automating machine learning [M]//Automated machine learning. Cham: Springer, 2019: 151–160.

    Chapter  Google Scholar 

  14. HOSANG J, BENENSON R, SCHIELE B. Learning non-maximum suppression [C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI: IEEE, 2017: 6469–6477.

    Google Scholar 

  15. DING R W, DAI L H, LI G P, et al. TDD-net: a tiny defect detection network for printed circuit boards [J]. CAAI Transactions on Intelligence Technology, 2019, 4(2): 110–116.

    Article  Google Scholar 

  16. TANG S L, HE F, HUANG X L, et al. Online PCB defect detector on a new PCB defect dataset [DB/OL]. (2019-02-17). https://arxiv.org/abs/1902.06197.

  17. DALAL N, TRIGGS B. Histograms of oriented gradients for human detection [C]//2005 IEEE Computer-Society Conference on Computer Vision and Pattern Recognition. San Diego, CA: IEEE, 2005: 886–893.

    Google Scholar 

  18. LOWE D G. Distinctive image features from scale-invariant keypoints [J]. International Journal of Computer Vision, 2004, 60(2): 91–110.

    Article  Google Scholar 

  19. BAY H, TUYTELAARS T, VAN GOOL L. SURF: speeded up robust features [M]//Computer vision — ECCV 2006. Berlin, Heidelberg: Springer, 2006: 404–417.

    Chapter  Google Scholar 

  20. LU Z S, HE Q Q, XIANG X G, et al. Defect detection of PCB based on Bayes feature fusion [J]. The Journal of Engineering, 2018, 2018(16): 1741–1745.

    Article  Google Scholar 

  21. BENEDEK C. Detection of soldering defects in printed circuit boards with hierarchical marked point processes [J]. Pattern Recognition Letters, 2011, 32(13): 1535–1543.

    Article  Google Scholar 

  22. GAIDHANE V H, HOTE Y V, SINGH V. An efficient similarity measure approach for PCB surface defect detection [J]. Pattern Analysis and Applications, 2018, 21(1): 277–289.

    Article  MathSciNet  Google Scholar 

  23. ZHANG C, SHI W, LI X F, et al. Improved bare PCB defect detection approach based on deep feature learning [J]. The Journal of Engineering, 2018, 2018(16): 1415–1420.

    Article  Google Scholar 

  24. DAI W T, MUJEEB A, ERDT M, et al. Soldering defect detection in automatic optical inspection [J]. Advanced Engineering Informatics, 2020, 43: 101004.

    Article  Google Scholar 

  25. BENJDIRA B, KHURSHEED T, KOUBAA A, et al. Car detection using unmanned aerial vehicles: Comparison between faster R-CNN and YOLOv3 [C]//2019 1st International Conference on Unmanned Vehicle Systems-Oman. Muscat: IEEE, 2019: 1–6.

    Google Scholar 

  26. LEI H W, WANG B, WU H H, et al. Defect detection for polymeric polarizer based on faster R-CNN [J]. Journal of Information Hiding and Multimedia Signal Processing, 2018, 9(6): 1414–1420.

    Google Scholar 

  27. LI Y T, HUANG H S, XIE Q S, et al. Research on a surface defect detection algorithm based on MobileNet-SSD [J]. Applied Sciences, 2018, 8(9): 1678.

    Article  Google Scholar 

  28. LI J Y, SU Z F, GENG J H, et al. Real-time detection of steel strip surface defects based on improved YOLO detection network [J]. IFAC-PapersOnLine, 2018, 51(21): 76–81.

    Article  Google Scholar 

  29. HOU W, WEI Y, GUO J, et al. Automatic detection of welding defects using deep neural network [J]. Journal of Physics: Conference Serie, 2017, 933: 012006.

    Google Scholar 

  30. LIN H, LI B, WANG X G, et al. Automated defect inspection of LED chip using deep convolutional neural network [J]. Journal of Intelligent Manufacturing, 2019, 30(6): 2525–2534.

    Article  Google Scholar 

  31. LIN J H, YAO Y, MA L, et al. Detection of a casting defect tracked by deep convolution neural network [J]. The International Journal of Advanced Manufacturing Technology, 2018, 97(1/2/3/4): 573–581.

    Article  Google Scholar 

  32. NASROLLAHI M, BOLOURIAN N, HAMMAD A. Concrete surface defect detection using deep neural network based on lidar scanning [C]//CSCE Annual Conference. Laval: CSCE, 2019: CON032.

    Google Scholar 

  33. MEI S, WANG Y D, WEN G J. Automatic fabric defect detection with a multi-scale convolutional denoising autoencoder network model [J]. Sensors, 2018, 18(4): 1064.

    Article  Google Scholar 

  34. ADIBHATLA V A, CHIH H C, HSU C C, et al. Defect detection in printed circuit boards using You-only-look-once convolutional neural networks [J]. Electronics, 2020, 9(9): 1547.

    Article  Google Scholar 

  35. ZHANG X, YANG Y H, HAN Z G, et al. Object class detection [J]. ACM Computing Surveys, 2013, 46(1): 1–53.

    Article  Google Scholar 

  36. SENGUPTA A, YE Y T, WANG R, et al. Going deeper in spiking neural networks: VGG and residual architectures [J]. Frontiers in Neuroscience, 2019, 13: 95.

    Article  Google Scholar 

  37. HENDRY, CHEN R C. Automatic License Plate Recognition via sliding-window darknet-YOLO deep learning [J]. Image and Vision Computing, 2019, 87: 47–56.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computer Engineering and Science, Shanghai University, Shanghai, 200444, China

    Xing Wu  (武 星), Qingfeng Zhang  (张庆丰) & Jianjia Wang  (王健嘉)

  2. Shanghai Institute for Advanced Communication and Data Science, Shanghai University, Shanghai, 200444, China

    Xing Wu  (武 星) & Jianjia Wang  (王健嘉)

  3. Shanghai Marine High-End Equipment Functional Platform Co., Ltd., Shanghai, 201306, China

    Junfeng Yao  (姚骏峰)

  4. Hong Kong Baptist University, Hong Kong, China

    Yike Guo  (郭毅可)

  5. Data Science Institute, Imperial College London, London, SW7 2AZ, UK

    Yike Guo  (郭毅可)

Authors
  1. Xing Wu  (武 星)
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qingfeng Zhang  (张庆丰)
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jianjia Wang  (王健嘉)
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Junfeng Yao  (姚骏峰)
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yike Guo  (郭毅可)
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xing Wu  (武 星).

Additional information

Foundation item: the Natural Science Foundation of Shanghai (No. 20ZR1420400), and the State Key Program of National Natural Science Foundation of China (No. 61936001)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wu, X., Zhang, Q., Wang, J. et al. Multiple Detection Model Fusion Framework for Printed Circuit Board Defect Detection. J. Shanghai Jiaotong Univ. (Sci.) 28, 717–727 (2023). https://doi.org/10.1007/s12204-022-2471-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12204-022-2471-0

Key words

关键词

CLC number

Document code

Search

Navigation