Skip to main content

Deep learning object detection applied to defect recognition of memory modules

The International Journal of Advanced Manufacturing Technology volume 121, pages 8433–8445 (2022)Cite this article

Abstract

For a long time, image inspection has been used for surface defect inspection of memory modules. However, its inspection accuracy still does not meet the requirements of mass production, especially for defect inspection of small parts. Deep learning algorithms can improve the inspection accuracy, but they require a considerable amount of actual production defect data. Therefore, in this study, real data for 70,000 pieces of memory modules were obtained during mass production to explore data pre-processing and data augmentation that meets the algorithms’ training needs. Due to limited data availability, it is necessary to collect an appropriate number of images and mark the fixed areas or features that represent the more frequently occurring defects in images. The software can learn by itself, speed up the operation, and correctly find the real defect position, effectively improving the detection accuracy so that the algorithm has the advantages of easy training and fast detection speed. The YOLOv5 algorithm has a better detection speed for smaller objects and uses the algorithm’s architectural characteristics to flexibly configure models with different complexities, thereby accelerating the convergence as well as simplifying the model architecture and accelerating the calculation speed. During the verification process, the average detection accuracy of production defects can reach 97.5%. It only takes an average of 0.5 s to detect each memory module picture, the yield rate of the production line per quarter is improved by up to 0.08%, and the false positive rate is reduced by 0.12%. In addition, it can improve the efficiency of personnel operations by nearly 40% and save up to 10,000 US dollars in production and operating costs per month.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26
Fig. 27
Fig. 28
Fig. 29
Fig. 30

Availability of data and material

All data generated or analyzed during this study are included in this published article.

Code availability

Not applicable.

References

  1. Hosang J, Benenson R, Dollár P, Schiele B (2015) What makes for effective detection proposals? https://arxiv.org/abs/1502.05082

  2. Anoop KP, Sarath NS, Kumar VS (2015) A review of PCB defect detection using image processing. https://www.ijeit.com/Vol%204/Issue%2011/IJEIT1412201505_31.pdf

  3. Lu H, Mehta D, Paradis O, Asadizanjani N, Tehranipoor M, Woodard DL (2020) FICS-PCB: a multi-modal image dataset for automated printed circuit board visual inspection. https://www.researchgate.net/publication/344475848_FICS-PCB_A_Multi-Modal_Image_Dataset_for_Automated_Printed_Circuit_Board_Visual_Inspection

  4. Lian J, Wang L, Liu T, Ding X, Yu Z (2021) Automatic visual inspection for printed circuit board via novel Mask R-CNN in smart city applications. https://www.sciencedirect.com/science/article/abs/pii/S2213138821000424

  5. Xin H, Chen Z, Wang B (2021) PCB electronic component defect detection method based on improved YOLOv4 algorithm. https://iopscience.iop.org/article/10.1088/1742-6596/1827/1/012167/pdf

  6. Alghassab MA (2022) Defect detection in printed circuit boards with pre-trained feature extraction methodology with convolution neural networks. https://www.techscience.com/cmc/v70n1/44420

  7. Ren S, He K, Girshick R, Sun J (2016) Faster R-CNN towards real-time object detection with region proposal networks. https://arxiv.org/abs/1506.01497

  8. Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: unified, real-time object detection. https://arxiv.org/abs/1506.02640

  9. Redmon J, Farhadi A (2016) YOLO9000: Better, faster, stronger. https://arxiv.org/abs/1612.08242

  10. Girshick R (2015) Fast R-CNN. https://arxiv.org/abs/1504.08083

  11. Bochkovskiy A, Wang CY, Liao HY (2020) YOLOv4: optimal speed and accuracy of object detection. https://arxiv.org/abs/2004.10934

  12. He K, Gkioxari G, Dollár P, Girshick R (2018) Mask R-CNN. https://arxiv.org/abs/1703.06870

  13. Sermanet P, Eigen D, Zhang X, Mathieu M, Fergus R, LeCun Y (2014) OverFeat: integrated recognition, localization and detection using convolutional networks. https://arxiv.org/abs/1312.6229

  14. Kisantal M, Wojna Z, Murawski J, Naruniec J, Cho K (2019) Augmentation for small object detection. https://arxiv.org/abs/1902.07296

  15. Wang CY, Liao HY, Wu YH, Chen PY, Hsieh JW, Yeh IH (2019) CSPNET: a new backbone that can enhance learning capability of CNN. https://arxiv.org/abs/1911.11929

  16. Huang Z, Wang J (2019) DC-SPP-YOLO: dense connection and spatial pyramid pooling based YOLO for object detection. https://arxiv.org/abs/1903.08589

  17. Lin TY, Goyal P, Girshick R, He K, Dollár P (2018) Focal loss for dense object detection. https://arxiv.org/abs/1708.02002

  18. Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu CY, Berg AC (2016) SSD (single shot multibox detector). https://arxiv.org/abs/1512.02325

  19. Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. https://arxiv.org/abs/1311.2524

  20. Tan M, Pang R, Le QV (2020) EfficientDet: scalable and efficient object detection. https://arxiv.org/abs/1911.09070

  21. Rezatofighi H, Tsoi N, Gwak J, Sadeghian A, Reid I, Savarese S (2019) Intersection over Union (IoU) for object detection. http://openaccess.thecvf.com/content_CVPR_2019/html/Rezatofighi_Generalized_Intersection_Over_Union_A_Metric_and_a_Loss_for_CVPR_2019_paper.html

  22. Redmon J, Farhadi A (2018) YOLOv3: an incremental improvement. https://arxiv.org/abs/1804.02767

  23. Rezatofighi H, Tsoi N, Gwak J, Sadeghian A, Reid I, Savarese S (2019) Generalized intersection over union: a metric and a loss for bounding box regression. https://arxiv.org/abs/1902.09630

  24. Mei S, Yang H, Yin Z (2018) An unsupervised-learning-based approach for automated defect inspection on textured surfaces. https://ieeexplore.ieee.org/document/8281622

  25. Weimer D, Scholz-Reiter B, Spitalnick M (2016) Design of deep convolutional neural network architectures for automated feature extraction in industrial inspection. https://www.sciencedirect.com/science/article/pii/S0007850616300725

  26. Tabernik D, Šela S, Skvarč J, Skočaj D (2019) Segmentation-based deep-learning approach for surface-defect detection. https://arxiv.org/abs/1903.08536

  27. Khalilian S, Hallaj Y, Balouchestani A, Karshenas H (2020) PCB defect detection using denoising convolutional autoencoders. https://arxiv.org/abs/2008.12589

  28. Mera C, Orozco-Alzate M, Branch J (2019) Incremental learning of concept drift in multiple instance learning for industrial visual inspection. https://www.sciencedirect.com/science/article/pii/S0166361519300466

  29. Ge C, Wang J, Wang J, Qi Q, Sun H, Liao J (2020) Towards automatic visual inspection: a weakly supervised learning method for industrial applicable object detection. https://dl.acm.org/doi/abs/10.1016/j.compind.2020.103232

  30. Sun J, Jia J, Shum HY (2004) Poisson matting. https://jiaya.me/all_final_papers/matting_siggraph04.pdf

  31. Cubuk ED, Zoph B, Mane D, Vasudevan V, Le QV (2019) AutoAugment:learning augmentation strategies from data. https://openaccess.thecvf.com/content_CVPR_2019/papers/Cubuk_AutoAugment_Learning_Augmentation_Strategies_From_Data_CVPR_2019_paper.pdf

  32. Lim S, Kim I, Kim T, Kim C (2019) Fast autoaugment. https://arxiv.org/abs/1905.00397

  33. Hu B, Wang J (2020) Detection of PCB surface defects with improved faster-RCNN and feature pyramid network. https://ieeexplore.ieee.org/iel7/6287639/8948470/09113299.pdf

Download references

Author information

Authors and Affiliations

  1. Department of Mechanical Engineering, National Taipei University of Technology, 1, Sec. 3, Zhongxiao E. Rd, Taipei, 10608, Taiwan

    Jung-Tang Huang & Chien-Hung Ting

Authors
  1. Jung-Tang Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chien-Hung Ting
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the study’s conception and design. Material preparation, data collection, and analysis were performed by JTH and CHT. The first draft of the manuscript was written by JTH, and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Jung-Tang Huang.

Ethics declarations

Ethics approval

The authors have no competing interests to declare that are relevant to the content of this article.

Consent to participate

All authors are involved in manuscript publication.

Consent for publication

All authors agree to publish the manuscript.

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Huang, JT., Ting, CH. Deep learning object detection applied to defect recognition of memory modules. Int J Adv Manuf Technol 121, 8433–8445 (2022). https://doi.org/10.1007/s00170-022-09716-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00170-022-09716-w

Keywords

  • Assembly optimization
  • Deep learning
  • Image recognition
  • Industrial inspection
  • Object detection