Skip to main content
SpringerLink
Log in

SD2Net: surface-mounted device detection network with convolution-free attention mechanism for printed circuit board integrity assurance

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

The accuracy of detecting and classifying surface-mounted devices (SMDs) on printed circuit boards (PCBs) is influenced by the complexity of the background and the variations in scale among the SMDs. Several object detection networks, such as the you only look once (YOLO) series-based networks and two-stage models like Mask R-CNN, have been developed to handle the detection of surface-mounted devices (SMDs) at various scales. However, the majority of these networks exhibit a complex structure with an abundance of parameters, which presents difficulties in deploying SMD detection networks in real-world application scenarios. To overcome this challenge, we introduce SD2Net, a convolution-free attention-based network specifically designed for accurate SMD detection. The front-end of SD2Net employs the proposed pyramid vision transformer (PVT) block-based backbone to extract SMD features in five stages, incorporating an attention mechanism. Subsequently, the extracted multi-scale features are fed into the feature pyramid network (FPN) for efficient feature fusion. In the back-end of SD2Net, the final stage integrates the VarifocalNet head module (VFH). By utilizing the Varifocal loss and generalized intersection over union (GIoU), it performs precise regression and computation of the localization and class mapping vector for SMDs. We conduct extensive experiments using two well-established publicly available datasets: FICS-PCB and PCB-WACV. The experimental results indicate the superior performance of our model in comparison to state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Availability of data and materials

PCB-WACE and FICS-PCB datasets that support the findings of this study are available from hyperlinks "https://ripl.cc.gatech.edu/data/pcb_wacv_2019.zip" and "https://www.trust-hub.org/#/data/fics-pcb".

Code Availability

Not applicable.

References

  1. Yang T, Yi X, Lu S, Johansson KH, Chai T (2021) Intelligent manufacturing for the process industry driven by industrial artificial intelligence. Eng 7(9):1224–1230

    Article  Google Scholar 

  2. Li J, Chen Y, Li W, Gu J (2022) Balancedyolov3: Addressing the imbalance problem of object detection in pcb assembly scene. Electron 11(8):1183

    Article  Google Scholar 

  3. An K, Zhang Y (2022) Lpvit: A transformer based model for pcb image classification and defect detection. IEEE Access 10:42542–42553

    Article  Google Scholar 

  4. Liu X, Hu J, Wang H, Zhang Z, Lu X, Sheng C, Song S, Nie J (2022) Gaussian-iou loss: Better learning for bounding box regression on pcb component detection. Expert Syst Appl 190:116178

    Article  Google Scholar 

  5. Liao HC, Lim ZY, Hu YX, Tseng HW (2018) Guidelines of automated optical inspection (aoi) system development. In: 2018 IEEE 3rd International Conference on Signal and Image Processing (ICSIP). IEEE, pp. 362–366

  6. Huanga X, Lib T, Chenc W (2022) Pcb assembly component recognition based on semantic segmentation and attention mechanism. Int Core J Eng 8(4):299–310

    Google Scholar 

  7. Zeng N, Wu P, Wang Z, Li H, Liu W, Liu X (2022) A small-sized object detection oriented multi-scale feature fusion approach with application to defect detection. IEEE Trans Instrum Meas 71:1–14

  8. Li J, Gu J, Huang Z, Wen J (2019) Application research of improved yolo v3 algorithm in pcb electronic component detection. Appl Sci 9(18):3750

    Article  Google Scholar 

  9. Mahalingam G, Gay KM, Ricanek K (2019) Pcb-metal: A pcb image dataset for advanced computer vision machine learning component analysis. In: 2019 16th International Conference on Machine Vision Applications (MVA). IEEE, pp. 1–5

  10. Silva LHdS, George OdA, Fernandes BJ, Bezerra BL, Lima EB, Oliveira SC (2019) Automatic optical inspection for defective pcb detection using transfer learning. In: 2019 IEEE Latin American Conference on Computational Intelligence (LA-CCI). IEEE, pp. 1–6

  11. Wang Z, Chen W, Li T, Zhang S, Xiong R (2022) Improved yolov3 detection method for pcb plug-in solder joint defects based on ordered probability density weighting and attention mechanism. AI Commun (Preprint) 1–16

  12. Xia B, Cao J, Wang C (2019) Ssim-net: Realtime pcb defect detection based on ssim and mobilenet-v3. In: 2019 2nd World Conference on Mechanical Engineering and Intelligent Manufacturing (WCMEIM). IEEE, pp. 756–759

  13. Lan Z, Hong Y, Li Y (2021) An improved yolov3 method for pcb surface defect detection. In: 2021 IEEE International Conference on Power Electronics, Computer Applications ICPECA). IEEE, pp. 1009–1015

  14. Cheong LK, Suandi SA, Rahman S (2019) Defects and components recognition in printed circuit boards using convolutional neural network. In: 10th International Conference on Robotics, Vision, Signal Processing and Power Applications. Springer, pp. 75–81

  15. Lian J, Wang L, Liu T, Ding X, Yu Z (2021) Automatic visual inspection for printed circuit board via novel mask r-cnn in smartcity applications. Sustain Energy Technol Assess 44:101032

    Google Scholar 

  16. Ren S, He K, Girshick R, Sun J (2015) Faster rcnn: Towards real-time object detection with region proposal networks. Adv Neural Inf Process Syst 28

  17. Wu B, Iandola F, Jin PH, Keutzer K (2017) Squeezedet: Unified, small, low power fully convolutional neural networks for real-time object detection for autonomous driving. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 129–137

  18. Pope R, Douglas S, Chowdhery A, Devlin J, Bradbury J, Levskaya A, Heek J, Xiao K, Agrawal S, Dean J (2022) Efficiently scaling transformer inference. arXiv:2211.05102

  19. Sandler M, Howard A, Zhu M, Zhmoginov A, Chen LC (2018) Mobilenetv2: Inverted residuals and linear bottlenecks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4510–4520

  20. Li J, Li W, Chen Y, Gu J (2021) A pcb electronic components detection network design based on effective receptive field size and anchor size matching. Comput Intell Neurosci 2021

  21. Wang W, Xie E, Li X, Fan DP, Song K, Liang D, Lu T, Luo P, Shao L (2021) Pyramid vision transformer: A versatile backbone for dense prediction without convolutions. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 568–578

  22. Lin TY, Dollár P, Girshick R, He K, Hariharan B, Belongie S (2017) Feature pyramid networks for object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2117–2125

  23. Zhang H, Wang Y, Dayoub F, Sunderhauf N (2021) Varifocalnet: An iou-aware dense object detector. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8514–8523

  24. Rezatofighi H, Tsoi N, Gwak J, Sadeghian A, Reid I, Savarese S (2019) Generalized intersection over union: A metric and a loss for bounding box regression. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 658–666

  25. Kuo CW, Ashmore JD, Huggins D, Kira Z (2019) Data-efficient graph embedding learning for pcb component detection. In: 2019 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, pp. 551–560

  26. Ding R, Dai L, Li G, Liu H (2019) Tdd-net: a tiny defect detection network for printed circuit boards. CAAI Trans Intell Technol 4(2):110–116

    Article  Google Scholar 

  27. Tang S, He F, Huang X, Yang J (2019) Online pcb defect detector on a new pcb defect dataset. arXiv:1902.06197

  28. Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X, Unterthiner T, Dehghani M, Minderer M, Heigold G, Gelly S, et al (2020) An image is worth 16x16 words: Transformers for image recognition at scale. arXiv:2010.11929

  29. Jiang M, Song L, Wang Y, Li Z, Song H (2022) Fusion of the yolov4 network model and visual attention mechanism to detect lowquality young apples in a complex environment. Precis Agricu 23(2):559–577

    Article  Google Scholar 

  30. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778

  31. Bochkovskiy A, Wang CY, Liao HYM (2020) Yolov4: Optimal speed and accuracy of object detection. arXiv:2004.10934

  32. Lu H, Mehta D, Paradis O, Asadizanjani N, Tehranipoor M, Woodard DL (2020) Ficspcb: A multi-modal image dataset for automated printed circuit board visual inspection. Cryptology ePrint Archive

  33. Chen Z, Yang C, Li Q, Zhao F, Zha ZJ, Wu F (2021) Disentangle your dense object detector. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 4939–4948

  34. Feng C, Zhong Y, Gao Y, Scott MR, Huang W (2021) Tood: Task-aligned one-stage object detection. In: 2021 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE Computer Society, pp. 3490–3499

  35. Zhu X, Su W, Lu L, Li B, Wang X, Dai J (2020) Deformable detr: Deformable transformers for end-to-end object detection. arXiv:2010.04159

  36. Ge Z, Liu S, Wang F, Li Z, Sun J (2021) Yolox: Exceeding yolo series in 2021. arXiv:2107.08430

  37. Redmon J, Farhadi A (2018) Yolov3: An incremental improvement. arXiv:1804.02767

  38. Sun P, Zhang R, Jiang Y, Kong T, Xu C, Zhan W, Tomizuka M, Li L, Yuan Z, Wang C, et al (2021) Sparse r-cnn: End-to-end object detection with learnable proposals. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14454–14463

  39. Xie S, Girshick R, Dollár P, Tu Z, He K (2017) Aggregated residual transformations for deep neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1492–1500

  40. Chauhan S, Singh M, Aggarwal AK (2023) Designing of optimal digital iir filter in themulti-objective framework using an evolutionary algorithm. Eng Appl Artif Intell 119:105803

    Article  Google Scholar 

Download references

Funding

No funding was received to assist with the preparation of this manuscript.

Author information

Authors and Affiliations

  1. School of Electrical and Electronic Engineering, Shanghai Institute of Technology, Shanghai, 201418, China

    Zhihao Wang & Xizhong Shen

Authors
  1. Zhihao Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xizhong Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Conceptualization: Z.W., X.S.; Methodology: Z.W.; Formal analysis and investigation: Z.W.; Writing - original draft preparation: Z.W.; Writing - review and editing: Z.W., X.S.;Resources: Z.W.; Supervision: X.S.

Corresponding author

Correspondence to Zhihao Wang.

Ethics declarations

Competing interests

The authors have no financial or proprietary interests in any material discussed in this article.

Ethics approval

Not applicable.

Consent to participate

Not applicable.

Consent for publication

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, Z., Shen, X. SD2Net: surface-mounted device detection network with convolution-free attention mechanism for printed circuit board integrity assurance. Appl Intell 53, 23582–23595 (2023). https://doi.org/10.1007/s10489-023-04800-4

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-023-04800-4

Keywords

Search

Navigation