For a long time, image inspection has been used for surface defect inspection of memory modules. However, its inspection accuracy still does not meet the requirements of mass production, especially for defect inspection of small parts. Deep learning algorithms can improve the inspection accuracy, but they require a considerable amount of actual production defect data. Therefore, in this study, real data for 70,000 pieces of memory modules were obtained during mass production to explore data pre-processing and data augmentation that meets the algorithms’ training needs. Due to limited data availability, it is necessary to collect an appropriate number of images and mark the fixed areas or features that represent the more frequently occurring defects in images. The software can learn by itself, speed up the operation, and correctly find the real defect position, effectively improving the detection accuracy so that the algorithm has the advantages of easy training and fast detection speed. The YOLOv5 algorithm has a better detection speed for smaller objects and uses the algorithm’s architectural characteristics to flexibly configure models with different complexities, thereby accelerating the convergence as well as simplifying the model architecture and accelerating the calculation speed. During the verification process, the average detection accuracy of production defects can reach 97.5%. It only takes an average of 0.5 s to detect each memory module picture, the yield rate of the production line per quarter is improved by up to 0.08%, and the false positive rate is reduced by 0.12%. In addition, it can improve the efficiency of personnel operations by nearly 40% and save up to 10,000 US dollars in production and operating costs per month.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Price excludes VAT (USA)
Tax calculation will be finalised during checkout.
Availability of data and material
All data generated or analyzed during this study are included in this published article.
Hosang J, Benenson R, Dollár P, Schiele B (2015) What makes for effective detection proposals? https://arxiv.org/abs/1502.05082
Anoop KP, Sarath NS, Kumar VS (2015) A review of PCB defect detection using image processing. https://www.ijeit.com/Vol%204/Issue%2011/IJEIT1412201505_31.pdf
Lu H, Mehta D, Paradis O, Asadizanjani N, Tehranipoor M, Woodard DL (2020) FICS-PCB: a multi-modal image dataset for automated printed circuit board visual inspection. https://www.researchgate.net/publication/344475848_FICS-PCB_A_Multi-Modal_Image_Dataset_for_Automated_Printed_Circuit_Board_Visual_Inspection
Lian J, Wang L, Liu T, Ding X, Yu Z (2021) Automatic visual inspection for printed circuit board via novel Mask R-CNN in smart city applications. https://www.sciencedirect.com/science/article/abs/pii/S2213138821000424
Xin H, Chen Z, Wang B (2021) PCB electronic component defect detection method based on improved YOLOv4 algorithm. https://iopscience.iop.org/article/10.1088/1742-6596/1827/1/012167/pdf
Alghassab MA (2022) Defect detection in printed circuit boards with pre-trained feature extraction methodology with convolution neural networks. https://www.techscience.com/cmc/v70n1/44420
Ren S, He K, Girshick R, Sun J (2016) Faster R-CNN towards real-time object detection with region proposal networks. https://arxiv.org/abs/1506.01497
Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: unified, real-time object detection. https://arxiv.org/abs/1506.02640
Redmon J, Farhadi A (2016) YOLO9000: Better, faster, stronger. https://arxiv.org/abs/1612.08242
Girshick R (2015) Fast R-CNN. https://arxiv.org/abs/1504.08083
Bochkovskiy A, Wang CY, Liao HY (2020) YOLOv4: optimal speed and accuracy of object detection. https://arxiv.org/abs/2004.10934
He K, Gkioxari G, Dollár P, Girshick R (2018) Mask R-CNN. https://arxiv.org/abs/1703.06870
Sermanet P, Eigen D, Zhang X, Mathieu M, Fergus R, LeCun Y (2014) OverFeat: integrated recognition, localization and detection using convolutional networks. https://arxiv.org/abs/1312.6229
Kisantal M, Wojna Z, Murawski J, Naruniec J, Cho K (2019) Augmentation for small object detection. https://arxiv.org/abs/1902.07296
Wang CY, Liao HY, Wu YH, Chen PY, Hsieh JW, Yeh IH (2019) CSPNET: a new backbone that can enhance learning capability of CNN. https://arxiv.org/abs/1911.11929
Huang Z, Wang J (2019) DC-SPP-YOLO: dense connection and spatial pyramid pooling based YOLO for object detection. https://arxiv.org/abs/1903.08589
Lin TY, Goyal P, Girshick R, He K, Dollár P (2018) Focal loss for dense object detection. https://arxiv.org/abs/1708.02002
Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu CY, Berg AC (2016) SSD (single shot multibox detector). https://arxiv.org/abs/1512.02325
Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. https://arxiv.org/abs/1311.2524
Tan M, Pang R, Le QV (2020) EfficientDet: scalable and efficient object detection. https://arxiv.org/abs/1911.09070
Rezatofighi H, Tsoi N, Gwak J, Sadeghian A, Reid I, Savarese S (2019) Intersection over Union (IoU) for object detection. http://openaccess.thecvf.com/content_CVPR_2019/html/Rezatofighi_Generalized_Intersection_Over_Union_A_Metric_and_a_Loss_for_CVPR_2019_paper.html
Redmon J, Farhadi A (2018) YOLOv3: an incremental improvement. https://arxiv.org/abs/1804.02767
Rezatofighi H, Tsoi N, Gwak J, Sadeghian A, Reid I, Savarese S (2019) Generalized intersection over union: a metric and a loss for bounding box regression. https://arxiv.org/abs/1902.09630
Mei S, Yang H, Yin Z (2018) An unsupervised-learning-based approach for automated defect inspection on textured surfaces. https://ieeexplore.ieee.org/document/8281622
Weimer D, Scholz-Reiter B, Spitalnick M (2016) Design of deep convolutional neural network architectures for automated feature extraction in industrial inspection. https://www.sciencedirect.com/science/article/pii/S0007850616300725
Tabernik D, Šela S, Skvarč J, Skočaj D (2019) Segmentation-based deep-learning approach for surface-defect detection. https://arxiv.org/abs/1903.08536
Khalilian S, Hallaj Y, Balouchestani A, Karshenas H (2020) PCB defect detection using denoising convolutional autoencoders. https://arxiv.org/abs/2008.12589
Mera C, Orozco-Alzate M, Branch J (2019) Incremental learning of concept drift in multiple instance learning for industrial visual inspection. https://www.sciencedirect.com/science/article/pii/S0166361519300466
Ge C, Wang J, Wang J, Qi Q, Sun H, Liao J (2020) Towards automatic visual inspection: a weakly supervised learning method for industrial applicable object detection. https://dl.acm.org/doi/abs/10.1016/j.compind.2020.103232
Sun J, Jia J, Shum HY (2004) Poisson matting. https://jiaya.me/all_final_papers/matting_siggraph04.pdf
Cubuk ED, Zoph B, Mane D, Vasudevan V, Le QV (2019) AutoAugment:learning augmentation strategies from data. https://openaccess.thecvf.com/content_CVPR_2019/papers/Cubuk_AutoAugment_Learning_Augmentation_Strategies_From_Data_CVPR_2019_paper.pdf
Lim S, Kim I, Kim T, Kim C (2019) Fast autoaugment. https://arxiv.org/abs/1905.00397
Hu B, Wang J (2020) Detection of PCB surface defects with improved faster-RCNN and feature pyramid network. https://ieeexplore.ieee.org/iel7/6287639/8948470/09113299.pdf
The authors have no competing interests to declare that are relevant to the content of this article.
Consent to participate
All authors are involved in manuscript publication.
Consent for publication
All authors agree to publish the manuscript.
Conflict of interest
The authors declare no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Huang, JT., Ting, CH. Deep learning object detection applied to defect recognition of memory modules. Int J Adv Manuf Technol 121, 8433–8445 (2022). https://doi.org/10.1007/s00170-022-09716-w
- Assembly optimization
- Deep learning
- Image recognition
- Industrial inspection
- Object detection