Skip to main content
SpringerLink
Log in

Development of an adaptive template for fast detection of lithographic patterns of light-emitting diode chips

  • ORIGINAL ARTICLE
  • Published:
The International Journal of Advanced Manufacturing Technology Aims and scope Submit manuscript

Abstract

With the expansion in light-emitting diode (LED) lighting market and technology, control of product quality has become the focus of LED development. To achieve high online production capacity, automated quality detection with object image has been employed for comparison using mostly the standard templates. However, the resulting poor fitting causes misjudgment of the detection system. This study proposes an adaptive template method to improve the system fitting, reduce the system misjudgment, and enhance the detection efficiency. The severely damaged LED chips were screened out based on their grayscale entropy indices and related coefficient indices, which enhanced the reliability of the adaptive template system and accelerated the overall system detection process. To overcome the displacement and scale changes of the lithographic patterns, the scale-invariant feature transform (SIFT) and Harris–Laplace methods were used for comparison. The scale-invariant feature transform (SIFT) and Harris–Laplace methods were used to search and compare the feature points of the chip patterns, with the aim to overcome the displacement and scale changes of the lithographic patterns and establish the adaptive template in real conditions. The fast correlation coefficient comparison method was compared with the adaptive template comparison method proposed in this study. The results showed that the detection accuracy of the new method was 98.36%, which is 15.79% more accurate than the fast correlation coefficient comparison method. In terms of time performance, the method proposed in this study took 0.08 s less to complete the partition defect template. Moreover, the average detection time per chip was reduced by another 1.4 s, which improved the efficiency by 30.43%. The adaptive template proposed in this study exclusively establish itself based on different lithographic patterns, thus improving the detection efficiency and accuracy of the LED industry, and raising its market competitiveness in the industry.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26
Fig. 27
Fig. 28
Fig. 29
Fig. 30
Fig. 31
Fig. 32
Fig. 33
Fig. 34
Fig. 35
Fig. 36
Fig. 37
Fig. 38
Fig. 39
Fig. 40

Similar content being viewed by others

Data availability

All data sets generated in this study are available from the corresponding author upon reasonable request.

References

  1. Lin H, Li B, Wang X, Shu Y, Niu S (2019) Automated defect inspection of LED chip using deep convolutional neural network. J Intell Manuf 30(6):2525–2534. https://doi.org/10.1007/s10845-018-1415-x

    Article  Google Scholar 

  2. Matsushita Y, Azuno T, Yamashita Y (2012) Fuel reduction in coastal squid jigging boats equipped with various combinations of conventional metal halide lamps and low-energy LED panels. Fish Res 125:14–19. https://doi.org/10.1016/j.fishres.2012.02.004

    Article  Google Scholar 

  3. Kuo CFJ, Hsu CTM, Liu ZX, Wu (2014) HC Automatic inspection system of LED chip using two-stages back-propagation neural network. J Intell Manuf 25(6):1235–1243. https://doi.org/10.1007/s10845-012-0725-7

    Article  Google Scholar 

  4. Eckstein HC, Zeitner UD, Leitel R, Stumpf M, Schleicher P, Bräuer A, Tünnermann A (2016) High dynamic grayscale lithography with an LED-based micro-image stepper. In: Optical Microlithography XXIX International Society for Optics and Photonics, p 9780 97800T. https://doi.org/10.1117/12.2219099

    Chapter  Google Scholar 

  5. Choi JW, MacDonald E, Wicker R (2010) Multi-material microstereolithography. Int J Adv Manuf Technol 49(5-8):543–551. https://doi.org/10.1007/s00170-009-2434-8

    Article  Google Scholar 

  6. Peng X, Bennamoun M, Ma Q, Lei Y, Zhang Q, Chen W (2010) Drift-correcting template update strategy for precision feature point tracking. Image Vis Comput 28(8):1280–1292. https://doi.org/10.1016/j.imavis.2010.01.007

    Article  Google Scholar 

  7. Ravishankar (1996) AR Future directions in industrial machine vision: a case study of semiconductor manufacturing applications. Image Vis Comput 14(1):3–19. https://doi.org/10.1016/0262-8856(95)01035-1

    Article  Google Scholar 

  8. Xiao J, Wei H (2014) Scale-invariant contour segment context in object detection. Image Vis Comput 32(12):1055–1066. https://doi.org/10.1016/j.imavis.2014.08.013

    Article  Google Scholar 

  9. Xie J, Pun CM, Pan Z, Gao H, Wang B (2019) Automatic medical image registration based on an integrated method combining feature and area information. Neural Process Lett 49(1):263–284. https://doi.org/10.1007/s11063-018-9808-6

    Article  Google Scholar 

  10. Wu WY, Hung CW, Yu WB (2013) The development of automated solder bump inspection using machine vision techniques. Int J Adv Manuf Technol 69(1-4):509–523. https://doi.org/10.1007/s00170-013-4994-x

    Article  Google Scholar 

  11. Kuo CH, Yang FC, Wing JJ, Yang CK (2006) Construction of 3D solder paste surfaces using multi-projection images. Int J Adv Manuf Technol 31(5-6):509–519

    Article  Google Scholar 

  12. Shi P, Qi Q, Qin Y, Scott PJ, Jiang X (2020) A novel learning-based feature recognition method using multiple sectional view representation. J Intell Manuf 31:1291–1309. https://doi.org/10.1007/s10845-020-01533-w

    Article  Google Scholar 

  13. Fröhlich HB, Grozmani N, Wolfschlaeger D, Goncalves AA, Schmitt RH (2020) Construction of small sets of reference images for feature descriptors fitting and their use in the multiclassification of parts in industry. Int J Adv Manuf Technol 108:105–116. https://doi.org/10.1007/s00170-020-05253-6

    Article  Google Scholar 

  14. Qin A, Guo L, You Z, Gao H, Wu X, Xiang S (2020) Research on automatic monitoring method of face milling cutter wear based on dynamic image sequence. Int J Adv Manuf Technol 110(11):3365–3376. https://doi.org/10.1007/s00170-020-05955-x

    Article  Google Scholar 

  15. Yoo JC, Han TH (2009) Fast normalized cross-correlation. Circuits Syst Signal Process 28(6):819–843. https://doi.org/10.1007/s00034-009-9130-7

    Article  MATH  Google Scholar 

  16. Li H, Lee WS, Wang K (2016) Immature green citrus fruit detection and counting based on fast normalized cross correlation (FNCC) using natural outdoor colour images. Precis Agric 17(6):678–697. https://doi.org/10.1007/s11119-016-9443-z

    Article  Google Scholar 

  17. Kuo CFJ, Tsai CH, Wang WR, Wu HC (2019) Automatic marking point positioning of printed circuit boards based on template matching technique. J Intell Manuf 30(2):671–685. https://doi.org/10.1007/s10845-016-1274-2

    Article  Google Scholar 

  18. Stefano DL, Marchionni M, Mattoccia S (2004) A fast area-based stereo matching algorithm. Image Vis Comput 22(12):983–1005. https://doi.org/10.1016/j.imavis.2004.03.009

    Article  Google Scholar 

  19. Wang Z, Gong S, Li D, Lu H (2017) Error analysis and improved calibration algorithm for LED chip localization system based on visual feedback. Int J Adv Manuf Technol 92(9-12):3197–3206. https://doi.org/10.1007/s00170-017-0390-2

    Article  Google Scholar 

  20. Perng DB, Liu HW, Chang CC (2011) Automated SMD LED inspection using machine vision. Int J Adv Manuf Technol 57(9-12):1065–1077. https://doi.org/10.1007/s00170-011-3338-y

    Article  Google Scholar 

  21. Wang L, Chen B, Xu P, Ren H, Fang X, Wan S (2020) Geometry consistency aware confidence evaluation for feature matching. Image Vis Comput 103:103984. https://doi.org/10.1016/j.imavis.2020.103984

    Article  Google Scholar 

  22. Wang XY, Hou LM, Wu J (2008) A feature-based robust digital image watermarking against geometric attacks. Image Vis Comput 26(7):980–989. https://doi.org/10.1016/j.imavis.2007.10.014

    Article  Google Scholar 

  23. Harris CG, Stephens M (1988) A combined corner and edge detector. In Alvey Vis Conf 15(50):10–5244. https://doi.org/10.5244/C.2.23

    Article  Google Scholar 

  24. Leutenegger S, Lynen S, Bosse M, Siegwart R, Furgale P (2015) Keyframe-based visual–inertial odometry using nonlinear optimization. Int J Robot Res 34(3):314–334. https://doi.org/10.1177/0278364914554813

    Article  Google Scholar 

  25. Lowry S, Sünderhauf N, Newman P, Leonard JJ, Cox D, Corke P, Milford MJ (2015) Visual place recognition: A survey. Int J Robot Res 32(1):1–19. https://doi.org/10.1109/TRO.2015.2496823

    Article  Google Scholar 

  26. Smith SM, Brady JM (1997) SUSAN—a new approach to low level image processing. Int J Comput Vis 23(1):45–78. https://doi.org/10.1023/A:1007963824710

    Article  Google Scholar 

  27. Chen LC, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2017) Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Trans Pattern Anal 40(4):834–848. https://doi.org/10.1109/TPAMI.2017.2699184

    Article  Google Scholar 

  28. Zhong F, He S, Li B (2017) Blob analyzation-based template matching algorithm for LED chip localization. Int J Adv Manuf Technol 93(1-4):55–63. https://doi.org/10.1007/s00170-015-7638-5

    Article  Google Scholar 

  29. Yan Z, Shi B, Sun L, Xiao J (2020) Surface defect detection of aluminum alloy welds with 3D depth image and 2D gray image. Int J Adv Manuf Technol 110(3):741–752. https://doi.org/10.1007/s00170-020-05882-x

    Article  Google Scholar 

  30. Lowe DG (1999) Object recognition from local scale-invariant features. In: Proceedings of the seventh IEEE international conference on computer vision, vol 2, pp 1150–1157. https://doi.org/10.1109/ICCV.1999.790410

    Chapter  Google Scholar 

  31. Schmidhuber J (2015) Deep learning in neural networks: an overview. Neural Netw 61:85–117. https://doi.org/10.1016/j.neunet.2014.09.003

    Article  Google Scholar 

  32. Sullivan GD, Baker KD, Anderson JADW (1985) Use of multiple difference-of-Gaussian filters to verify geometric models. Image Vis Comput 3(4):192–197. https://doi.org/10.1016/0262-8856(85)90007-1

    Article  Google Scholar 

  33. Se S, Lowe DG, Little JJ (2005) Vision-based global localization and mapping for mobile robots. IEEE Trans Robot 21(3):364–375. https://doi.org/10.1109/TRO.2004.839228

    Article  Google Scholar 

  34. Mikolajczyk K, Schmid C (2001) Indexing based on scale invariant interest points. In: Proceedings Eighth IEEE International Conference on Computer Vision ICCV, vol 1, pp 525–531. https://doi.org/10.1109/ICCV.2001.937561

    Chapter  Google Scholar 

  35. Dorkó G, Schmid C (2003) Selection of scale-invariant parts for object class recognition. ICCV 3:634. https://doi.org/10.1109/ICCV.2003.1238407

    Article  Google Scholar 

  36. Mikolajczyk K, Schmid C (2004) Scale & affine invariant interest point detectors. Int J Comput Vis 60(1):63–86. https://doi.org/10.1023/B:VISI.0000027790.02288.f2

    Article  Google Scholar 

  37. Gordo A, Almazán J, Revaud J, Larlus D (2016) Deep image retrieval: learning global representations for image search. In: European conference on computer vision, pp 241–257. https://doi.org/10.1007/978-3-319-46466-4_15

    Chapter  Google Scholar 

  38. Korytkowski M, Rutkowski L, Scherer R (2016) Fast image classification by boosting fuzzy classifiers. Inf Sci 327:175–182. https://doi.org/10.1016/j.ins.2015.08.030

    Article  MathSciNet  Google Scholar 

  39. Zhang J, Chen Q, Sun Q, Sun H, Xia D (2011) A highly repeatable feature detector: improved Harris–Laplace. Multimed Tools Appl 52(1):175–186. https://doi.org/10.1007/s11042-010-0471-9

    Article  Google Scholar 

  40. Fawcett T (2006) An introduction to ROC analysis. Pattern Recogn Lett 27(8):861–874. https://doi.org/10.1016/j.patrec.2005.10.010

    Article  MathSciNet  Google Scholar 

  41. Chen D, Wang P, Yue L, Zhang Y, Jia T (2020) Anomaly detection in surveillance video based on bidirectional prediction. Image Vision Comut 98:103915. https://doi.org/10.1016/j.imavis.2020.103915

    Article  Google Scholar 

  42. Mason SE, Nicolay CR, Darzi (2015) A The use of lean and six sigma methodologies in surgery: a systematic review. Surgeon 13(2):91–100. https://doi.org/10.1016/j.surge.2014.08.002

    Article  Google Scholar 

  43. Kwak YH, Anbari FT (2006) Benefits, obstacles, and future of six sigma approach. Technovation 26(5-6):708–715. https://doi.org/10.1016/j.technovation.2004.10.003

    Article  Google Scholar 

Download references

Funding

The research was supported by the Ministry of Science and Technology of the Republic of China under Grant No. 109-2221-E-011-149.

Author information

Authors and Affiliations

  1. Department of Materials Science and Engineering, National Taiwan University of Science and Technology, Taipei, 106, Taiwan

    Wei-Han Weng, Cheng-Yu Hung & Chung-Feng Jeffrey Kuo

  2. Photonics Automation Division, MPI Corporation, Chu-Pei City, Hsinchu County, 302, Taiwan

    Chen-Yang Tsai

Authors
  1. Wei-Han Weng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chen-Yang Tsai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Cheng-Yu Hung
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chung-Feng Jeffrey Kuo
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Conceptualization, Chung-Feng Jeffrey Kuo; methodology, Cheng-Yu Hung and Wei-Han Weng; validation, Chen-Yang Tsa and Cheng-Yu Hung; formal analysis, Chung-Feng Jeffrey Kuo and Chen-Yang Tsa; data interpretation, Wei-Han Weng and Cheng-Yu Hung; writing—original draft preparation, Cheng-Yu Hung; writing—review and editing, Chung-Feng Jeffrey Kuo; visualization, Chen-Yang Tsai; funding acquisition, Chung-Feng Jeffrey Kuo.

All authors have read and agreed to the published version of the manuscript.

Corresponding author

Correspondence to Chung-Feng Jeffrey Kuo.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Weng, WH., Tsai, CY., Hung, CY. et al. Development of an adaptive template for fast detection of lithographic patterns of light-emitting diode chips. Int J Adv Manuf Technol 117, 3297–3321 (2021). https://doi.org/10.1007/s00170-021-07774-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00170-021-07774-0

Keywords

Search

Navigation