Skip to main content

Multiple Detection Model Fusion Framework for Printed Circuit Board Defect Detection

Abstract

The printed circuit board (PCB) is an indispensable component of electronic products, which determines the quality of these products. With the development and advancement of manufacturing technology, the layout and structure of PCB are getting complicated. However, there are few effective and accurate PCB defect detection methods. There are high requirements for the accuracy of PCB defect detection in the actual production environment, so we propose two PCB defect detection frameworks with multiple model fusion including the defect detection by multi-model voting method (DDMV) and the defect detection by multi-model learning method (DDML). With the purpose of reducing wrong and missing detection, the DDMV and DDML integrate multiple defect detection networks with different fusion strategies. The effectiveness and accuracy of the proposed framework are verified with extensive experiments on two open-source PCB datasets. The experimental results demonstrate that the proposed DDMV and DDML are better than any other individual state-of-the-art PCB defect detection model in F1-score, and the area under curve value of DDML is also higher than that of any other individual detection model. Furthermore, compared with DDMV, the DDML with an automatic machine learning method achieves the best performance in PCB defect detection, and the F1-score on the two datasets can reach 99.7% and 95.6% respectively.

This is a preview of subscription content, access via your institution.

References

  1. CHAUDHARY V, DAVE I R, UPLA K P. Automatic visual inspection of printed circuit board for defect detection and classification [C]//2017 International Conference on Wireless Communications, Signal Processing and Networking. Chennai: IEEE, 2017: 732–737.

    Google Scholar 

  2. ZHU JH, WU A, LIU X P. Printed circuit board defect visual detection based on wavelet denoising [J]. IOP Conference Series: Materials Science and Engineering, 2018, 392: 062055.

    Article  Google Scholar 

  3. KUO C F J, FANG T Y, LEE C L, et al. Automated optical inspection system for surface mount device light emitting diodes [J]. Journal of Intelligent Manufacturing, 2019, 30(2): 641–655.

    Article  Google Scholar 

  4. KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks [J]. Communications of the ACM, 2017, 60(6): 84–90.

    Article  Google Scholar 

  5. WU X, ZHONG M Y, GUO Y K, et al. The assessment of small bowel motility with attentive deformable neural network [J]. Information Sciences, 2020, 508: 22–32.

    Article  Google Scholar 

  6. LIU W, ANGUELOV D, ERHAN D, et al. SSD: Single shot multibox detector [M]//Computer vision -ECCV 2016. Cham: Springer, 2016: 21–37.

    Chapter  Google Scholar 

  7. REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: Unified, real-time object detection [C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV: IEEE, 2016: 779–788.

    Google Scholar 

  8. LIN T Y, DOLLÁR P, GIRSHICK R, et al. Feature pyramid networks for object detection [C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI: IEEE, 2017: 936–944.

    Google Scholar 

  9. JIANG H Z, LEARNED-MILLER E. Face detection with the faster R-CNN [C]//2017 12th IEEE International Conference on Automatic Face & Gesture Recognition. Washington, DC, USA: IEEE, 2017: 650–657.

    Google Scholar 

  10. WANG Y, LUO X B, DING L, et al. Detection based visual tracking with convolutional neural network [J]. Knowledge-Based Systems, 2019, 175: 62–71.

    Article  Google Scholar 

  11. WEI H, YANG C Z, YU Q. Efficient graph-based search for object detection [J]. Information Sciences, 2017, 385/386: 395–414.

    Article  Google Scholar 

  12. BRIA A, MARROCCO C, MOLINARA M, et al. An effective learning strategy for cascaded object detection [J]. Information Sciences, 2016, 340/341: 17–26.

    MathSciNet  Article  Google Scholar 

  13. OLSON R S, MOORE J H. TPOT: A tree-based pipeline optimization tool for automating machine learning [M]//Automated machine learning. Cham: Springer, 2019: 151–160.

    Chapter  Google Scholar 

  14. HOSANG J, BENENSON R, SCHIELE B. Learning non-maximum suppression [C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI: IEEE, 2017: 6469–6477.

    Google Scholar 

  15. DING R W, DAI L H, LI G P, et al. TDD-net: a tiny defect detection network for printed circuit boards [J]. CAAI Transactions on Intelligence Technology, 2019, 4(2): 110–116.

    Article  Google Scholar 

  16. TANG S L, HE F, HUANG X L, et al. Online PCB defect detector on a new PCB defect dataset [DB/OL]. (2019-02-17). https://arxiv.org/abs/1902.06197.

  17. DALAL N, TRIGGS B. Histograms of oriented gradients for human detection [C]//2005 IEEE Computer-Society Conference on Computer Vision and Pattern Recognition. San Diego, CA: IEEE, 2005: 886–893.

    Google Scholar 

  18. LOWE D G. Distinctive image features from scale-invariant keypoints [J]. International Journal of Computer Vision, 2004, 60(2): 91–110.

    Article  Google Scholar 

  19. BAY H, TUYTELAARS T, VAN GOOL L. SURF: speeded up robust features [M]//Computer vision — ECCV 2006. Berlin, Heidelberg: Springer, 2006: 404–417.

    Chapter  Google Scholar 

  20. LU Z S, HE Q Q, XIANG X G, et al. Defect detection of PCB based on Bayes feature fusion [J]. The Journal of Engineering, 2018, 2018(16): 1741–1745.

    Article  Google Scholar 

  21. BENEDEK C. Detection of soldering defects in printed circuit boards with hierarchical marked point processes [J]. Pattern Recognition Letters, 2011, 32(13): 1535–1543.

    Article  Google Scholar 

  22. GAIDHANE V H, HOTE Y V, SINGH V. An efficient similarity measure approach for PCB surface defect detection [J]. Pattern Analysis and Applications, 2018, 21(1): 277–289.

    MathSciNet  Article  Google Scholar 

  23. ZHANG C, SHI W, LI X F, et al. Improved bare PCB defect detection approach based on deep feature learning [J]. The Journal of Engineering, 2018, 2018(16): 1415–1420.

    Article  Google Scholar 

  24. DAI W T, MUJEEB A, ERDT M, et al. Soldering defect detection in automatic optical inspection [J]. Advanced Engineering Informatics, 2020, 43: 101004.

    Article  Google Scholar 

  25. BENJDIRA B, KHURSHEED T, KOUBAA A, et al. Car detection using unmanned aerial vehicles: Comparison between faster R-CNN and YOLOv3 [C]//2019 1st International Conference on Unmanned Vehicle Systems-Oman. Muscat: IEEE, 2019: 1–6.

    Google Scholar 

  26. LEI H W, WANG B, WU H H, et al. Defect detection for polymeric polarizer based on faster R-CNN [J]. Journal of Information Hiding and Multimedia Signal Processing, 2018, 9(6): 1414–1420.

    Google Scholar 

  27. LI Y T, HUANG H S, XIE Q S, et al. Research on a surface defect detection algorithm based on MobileNet-SSD [J]. Applied Sciences, 2018, 8(9): 1678.

    Article  Google Scholar 

  28. LI J Y, SU Z F, GENG J H, et al. Real-time detection of steel strip surface defects based on improved YOLO detection network [J]. IFAC-PapersOnLine, 2018, 51(21): 76–81.

    Article  Google Scholar 

  29. HOU W, WEI Y, GUO J, et al. Automatic detection of welding defects using deep neural network [J]. Journal of Physics: Conference Serie, 2017, 933: 012006.

    Google Scholar 

  30. LIN H, LI B, WANG X G, et al. Automated defect inspection of LED chip using deep convolutional neural network [J]. Journal of Intelligent Manufacturing, 2019, 30(6): 2525–2534.

    Article  Google Scholar 

  31. LIN J H, YAO Y, MA L, et al. Detection of a casting defect tracked by deep convolution neural network [J]. The International Journal of Advanced Manufacturing Technology, 2018, 97(1/2/3/4): 573–581.

    Article  Google Scholar 

  32. NASROLLAHI M, BOLOURIAN N, HAMMAD A. Concrete surface defect detection using deep neural network based on lidar scanning [C]//CSCE Annual Conference. Laval: CSCE, 2019: CON032.

    Google Scholar 

  33. MEI S, WANG Y D, WEN G J. Automatic fabric defect detection with a multi-scale convolutional denoising autoencoder network model [J]. Sensors, 2018, 18(4): 1064.

    Article  Google Scholar 

  34. ADIBHATLA V A, CHIH H C, HSU C C, et al. Defect detection in printed circuit boards using You-only-look-once convolutional neural networks [J]. Electronics, 2020, 9(9): 1547.

    Article  Google Scholar 

  35. ZHANG X, YANG Y H, HAN Z G, et al. Object class detection [J]. ACM Computing Surveys, 2013, 46(1): 1–53.

    Article  Google Scholar 

  36. SENGUPTA A, YE Y T, WANG R, et al. Going deeper in spiking neural networks: VGG and residual architectures [J]. Frontiers in Neuroscience, 2019, 13: 95.

    Article  Google Scholar 

  37. HENDRY, CHEN R C. Automatic License Plate Recognition via sliding-window darknet-YOLO deep learning [J]. Image and Vision Computing, 2019, 87: 47–56.

    Article  Google Scholar 

Download references

Funding

the Natural Science Foundation of Shanghai (No. 20ZR1420400), and the State Key Program of National Natural Science Foundation of China (No. 61936001)

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Xing Wu.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Wu, X., Zhang, Q., Wang, J. et al. Multiple Detection Model Fusion Framework for Printed Circuit Board Defect Detection. J. Shanghai Jiaotong Univ. (Sci.) (2022). https://doi.org/10.1007/s12204-022-2471-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s12204-022-2471-0

Key words

  • printed circuit board (PCB)
  • defect detection
  • model fusion
  • object detection model

CLC number

  • TP 399

Document code

  • A