Pattern Analysis and Applications

, Volume 17, Issue 4, pp 863–881 | Cite as

Evaluating the reliability level of virtual metrology results for flexible process control: a novelty detection-based approach

  • Pilsung KangEmail author
  • Dongil Kim
  • Sungzoon Cho
Industrial and Commercial Application


The purpose of virtual metrology (VM) in semiconductor manufacturing is to support process monitoring and quality control by predicting the metrological values of every wafer without an actual metrology process, based on process sensor data collected during the operation. Most VM-based quality control schemes assume that the VM predictions are always accurate, which in fact may not be true due to some unexpected variations that can occur during the process. In this paper, therefore, we propose a means of evaluating the reliability level of VM prediction results based on novelty detection techniques, which would allow flexible utilization of the VM results. Our models generate a high-reliability score for a wafer’s VM prediction only when its process sensor values are found to be consistent with those of the majority of wafers that are used in model building; otherwise, a low-reliability score is returned. Thus, process engineers can selectively utilize VM results based on their reliability level. Experimental results show that our reliability generation models are effective; the VM results for wafers with a high level of reliability were found to be much more accurate than those with a low level.


Virtual metrology Reliability level Novelty detection Semiconductor Process monitoring 



The first author was supported by the research program funded by the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (NRF-2011-0021893) and by the Ministry of Science, ICT, and Future Planning (NRF-2014R1A1A1004648). The dataset used in this paper can be available upon request.


  1. 1.
    Angiulli F, Pizzuti C (2005) Outlier mining in large high-dimensional data sets. IEEE Trans Knowl Data Eng 17(2):203–215MathSciNetCrossRefGoogle Scholar
  2. 2.
    Ankerst M, Breunig MM, Kriegel HP, Sander J (1999) OPTICS: Ordering points to identify the clustering structure. In: Proceedings of the 1999 ACM SIGMOD International Conference on Management of Data, New York, NY, USA, pp 49–60Google Scholar
  3. 3.
    Barnett V, Lewis T (1994) Outliers in statistical data. Wiley and Sons, New YorkzbMATHGoogle Scholar
  4. 4.
    Besnard J, Toprac A (2006) Wafer-to-wafer virtual metrology applied to run-to-run control. In: Proceedings of the Third ISMI Symposium on Manufacturing Effectiveness, Austin, TX, USAGoogle Scholar
  5. 5.
    Bishop CM (1995) Neural networks for pattern recognition. Oxford University Press, New YorkGoogle Scholar
  6. 6.
    Bode C, Ko B, Edgar T (2004) Run-to-run control and performance monitoring of overlay in semiconductor manufacturing. Control Eng Pract 12(7):893–900CrossRefGoogle Scholar
  7. 7.
    Boutsinas B, Tasoulis DK, Vrahatis MN (2006) Estimating the number of clusters using a windowing technique. Pattern Recogn Image Anal 16(2):143–154CrossRefGoogle Scholar
  8. 8.
    Breunig MM, Kriegel HP, Ng RT, Sander J (2000) LOF: Identifying density-based local outliers. In: Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data, Dallas, Texas, USA, pp 93–104Google Scholar
  9. 9.
    Burge P, Shawe-Taylor J (1997) Detecting cellular fraud using adaptive prototypes. In: Proceedings of AAAI-97 Workshop on AI Approaches to Fraud Detection and Risk Management, Rhode Island, USA, pp 9–13Google Scholar
  10. 10.
    Castillo DE, Yeh JY (1998) An adaptive run-to-run optimizing controller for linear and nonlinear semiconductor processes. IEEE Trans Semicond Manuf 11(2):285–295CrossRefGoogle Scholar
  11. 11.
    Chang YJ, Kang Y, Hsu CL, Chang CT, Chan T (2006) Virtual metrology techniques for semiconductor manufacturing. In: Proceedings of the International Joint Conference on Neural Networks (IJCNN 2006), Vancouver, BC, Canada, pp 5289–5293Google Scholar
  12. 12.
    Chemali CE, Freudenberg J, Hankinson M, Bendik JJ (2004) Run-to-run critical dimension and sidewall angle lithography control using the PROLITH simulator. IEEE Trans Semicond Manuf 17(3):388–401CrossRefGoogle Scholar
  13. 13.
    Chen P, Wu S, Lin J, Ko F, Lo H, Wang J (2005) Virtual metrology: a solution for wafer to wafer advanced process control. In: Proceedings of International Symposium on Semiconductor Manufacturing (ISSM 2005), San Jose, CA, USA, pp 155–157Google Scholar
  14. 14.
    Chen YT, Yang HC, Cheng FT (2006) Multivariate simulation assessment for virtual metrology. In: Proceedings of IEEE International Conference on Robotics and Automation (ICRA 2006), Orlando, FL, USA, pp 1048–1053Google Scholar
  15. 15.
    Cheng CY, Cheng FT (2005) Application development of virtual metrology in semiconductor industry. In: Proceedings of The 31st Annual Conference of IEEE Industrial Electronics Society (IECON 2005), Raleigh, NC, USA, pp 124–129Google Scholar
  16. 16.
    Cheng FT, Chen YT, Su YC, Zeng DL (2008) Evaluating reliance level of a virtual metrology system. IEEE Trans Semicond Manuf 21(1):92–103CrossRefGoogle Scholar
  17. 17.
    Cheng FT, Chang JC, Huang HC, Kao CA, Chen YL, Peng JL (2011) Benefit model of virtual metrology and integrating AVM into MES. IEEE Trans Semicond Manuf 24(2):261–272CrossRefGoogle Scholar
  18. 18.
    Cheng FT, Huang HC, Kao CA (2012) Developing an automatic virtual metrology system. IEEE Trans Autom Sci Eng 9(1):181–188CrossRefGoogle Scholar
  19. 19.
    Cho SJ, Hermsmeier MA (2002) Genetic algorithm guided selection: variable selection and subset selection. J Chem Inf Model 42(4):927–936CrossRefGoogle Scholar
  20. 20.
    Cyganek B (2012) One-class support vector ensembles for image segmentation and classification. J Math Imaging Vis 42(2–3):103–117MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    Duda RO, Hart PE, Stork DG (2001) Pattern classification. Wiley, New YorkzbMATHGoogle Scholar
  22. 22.
    Ester M, Kriegel HP, Sander J, Xu X (1996) A density-based algorithm for discovering clusters in large spatial databases with noise. Second international conference on knowledge discovery and data mining. AAAI Press, Menlo Park, pp 226–231Google Scholar
  23. 23.
    Freund Y, Schapire RE (1997) A decision-theoretic generalization of on-line learning and an application to boosting. J Comput Syst Sci 55(1):119–139MathSciNetCrossRefzbMATHGoogle Scholar
  24. 24.
    Gergeret F, Gall G (2003) Yield improvement using statistical analysis of process data. IEEE Trans Semicond Manuf 16(3):535–542CrossRefGoogle Scholar
  25. 25.
    Guha S, Rastogi R, Shim K (1998) CURE: An efficient clustering algorithm for large databases. In: Proceedings of ACM SIGMOD Conference on Management of Data, Seattle, WA, USA, pp 73-84Google Scholar
  26. 26.
    Guha S, Rastogi R, Shim K (2000) ROCK: a robust clustering algorithm for categorical attributes. Inf Syst 25(5):345–366CrossRefGoogle Scholar
  27. 27.
    Harmeling S, Dornhege G, Tax D, Meinecke F, Müller KR (2006) From outliers to prototypes: ordering data. Neurocomputing 69(13–15):1608–1618CrossRefGoogle Scholar
  28. 28.
    Hastie T, Tibshirani R, Friedman J (2002) The element of statistical learning: data mining, inference, and prediction. Springer-Verlag, New YorkGoogle Scholar
  29. 29.
    Hawkins DM (1980) Identification of outliers. Chapman & Hall, LondonCrossRefzbMATHGoogle Scholar
  30. 30.
    Jarvis RM, Goodacre R (2004) Genetic algorithm optimization for pre-processing and variable selection of spectroscopic data. Bioinformatics 21(7):860–868CrossRefGoogle Scholar
  31. 31.
    Jiang H, Liu G, Xiao X, Mei C, Ding Y, Yu S (2012) Monitoring of solid-state fermentation of wheat straw in a pilot scale using FT-NIR spectroscopy and support vector data description. Microchem J 102:68–74CrossRefGoogle Scholar
  32. 32.
    Kang P, Cho S (2008) Locally linear reconstruction for instance-based learning. Pattern Recogn 41(11):3507–3518CrossRefzbMATHGoogle Scholar
  33. 33.
    Kang P, Cho S (2009) A hybrid novelty score and its use in keystroke dynamics-based user authentication. Pattern Recogn 42(11):3115–3127CrossRefzbMATHGoogle Scholar
  34. 34.
    Karypis G, Han EH, Kumar V (1999) Chameleon: hierarchical clustering using dynamic modeling. IEEE Comput Mag 32(8):68–75CrossRefGoogle Scholar
  35. 35.
    Khan A, Moyne J, Tilbury D (2007) An approach for factory-wide control utilizing virtual metrology. IEEE Trans Semicond Manuf 20(4):364–375CrossRefGoogle Scholar
  36. 36.
    Khazai S, Safari A, Mojaradi B, Homayouni S (2012) Improving the SVDD approach to hyperspectral image classification. IEEE Geosci Remote Sens Lett 9(4):594–598CrossRefGoogle Scholar
  37. 37.
    Knorr EM, Ng RT, Tucakov V (2000) Distance-based outliers: algorithms and applications. VLDB J 8(3–4):237–253CrossRefGoogle Scholar
  38. 38.
    Krawczyk B, Filipczuk P (2013) Cytological image analysis with firely nuclei detection and hybrid one-class classification decomposition. Eng Appl Artif Intell. doi: 10.1016/j.engappai.2013.09.017 Google Scholar
  39. 39.
    Krawczyk B, Woźniak M (2014) Diversity measures for one-class classifier ensembles. Neurocomputing 126:36–44CrossRefGoogle Scholar
  40. 40.
    Limanond S, Si J, Tsakalis K (1998) Monitoring and control of semiconductor manufacturing processes. IEEE Control Syst Mag 18(6):46–58CrossRefGoogle Scholar
  41. 41.
    Lin TH, Hung MT, Lin RC, Cheng FT (2006) A virtual metrology scheme for predicting CVD thickness in semiconductor manufacturing. In: Proceedings of IEEE International Conference on Robotics and Automation (ICRA 2006), Orlando, FL, USA, pp 1054–1059Google Scholar
  42. 42.
    Lynn S, Ringwood J, MacGrearailt N (2010) Weighted windowed PLS models for virtual metrology of an industrial plasma etch process. In: Proceedings of IEEE International Conference on Industrial Technology (ICIT 2010), Vina del Mar, Chile, pp 309–314Google Scholar
  43. 43.
    Mũnoz-Marí J, Bovolo F, Gomez-Chova L, Bruzzone L, Camp-Valls G (2010) Semisupervised one-class support vector machines for classification of remote sensing data. IEEE Trans Geosci Remote Sens 48(8):3188–3197CrossRefGoogle Scholar
  44. 44.
    McLachlan GJ, Peel D (2000) Finite mixture models. Wiley and Sons, New YorkCrossRefzbMATHGoogle Scholar
  45. 45.
    Müller KR, Mika S, Rätsch G, Tsuda K, Schölkopf B (2001) An introduction to kernel-based learning algorithms. IEEE Trans Neural Networks 12(2):181–201CrossRefGoogle Scholar
  46. 46.
    Pang TH, Sheng BQ, Wong DSH, Jang SS (2011) A virtual metrology system for predicting end-of-line electrical properties using MANCOVA model with tools clustering. IEEE Trans Ind Inf 7(2):187–195CrossRefGoogle Scholar
  47. 47.
    Park SJ, Lee MS, Shin SY, Cho KH, Lim JT, Cho BS, Jei YH, Kim MK, Park CH (2005) Run-to-run overlay control of steppers in semiconductor manufacturing systems. IEEE Trans Semicond Manuf 18(4):605–613CrossRefGoogle Scholar
  48. 48.
    Qin S, Cherry G, Good R, Wang J, Harrison C (2006) Semiconductor manufacturing process control and monitoring: a fab-wide framework. J Process Control 16(3):179–191CrossRefGoogle Scholar
  49. 49.
    Ramaswamy S, Rastogi R, Shim K (2000) Efficient algorithms for mining outliers from large data sets. In: Proceedings of International Conference on Management of Data (SIGMOD 2000), Dallas, TX, USAGoogle Scholar
  50. 50.
    Ross SM (2004) Introduction to probability and statistic for engineers and scientists. Academic Press, San DiegoGoogle Scholar
  51. 51.
    Sachs E, Hu A, Ingolfsson A (1995) Run by run process control: combining SPC and feedback control. IEEE Trans Semicond Manuf 8(1):26–43CrossRefGoogle Scholar
  52. 52.
    Schölkopf B, Platt JC, Shawe-Taylor J, Smola AJ (2001) Estimating the support of a high-dimensional distribution. Neural Comput 13(7):1443–1471CrossRefzbMATHGoogle Scholar
  53. 53.
    Shawe-Taylor J, Cristianini N (2004) Kernel methods for pattern analysis. Cambridge University Press, CambridgeCrossRefGoogle Scholar
  54. 54.
    Spanos C, Guo HF, Miller A, Levine-Parril J (1992) Real-time statistical process control using tool data. IEEE Trans Semicond Manuf 5(4):308–318CrossRefGoogle Scholar
  55. 55.
    Su AJ, Jeng JC, Huang HP, Yu CC, Hung SY, Chao CK (2007) Control relevant issues in semiconductor manufacturing: overview with some new results. Control Eng Pract 15(10):1268–1279CrossRefGoogle Scholar
  56. 56.
    Su YC, Lin TH, Cheng FT, Wu WM (2008) Accuracy and real-time considerations for implementing various virtual metrology algorithms. IEEE Trans Semicond Manuf 21(3):426–434CrossRefGoogle Scholar
  57. 57.
    Tax D (2001) One-class classification: Concept-learning in the absence of counterexamples. PhD thesis, Delft University of Technology, URL
  58. 58.
    Tax D, Duin R (1999) Support vector domain description. Pattern Recogn Lett 20(11–13):1191–1199CrossRefGoogle Scholar
  59. 59.
    Tax D, Duin R (2005) Characterizing one-class datasets. In: Proceedings of the Sixteenth Annual Symposium of the Pattern Recognition Association of South Africa, Langebaan, South Africa, pp 21–26Google Scholar
  60. 60.
    Vapnik V (1998) Statistical learning theory. Wiley and Sons, New YorkzbMATHGoogle Scholar
  61. 61.
    Wang XA, Mahajan R (1996) Artificial neural network model-based run-to-run process controller. IEEE Trans Compon Packag Manuf Technol part C 19(1):19–26CrossRefGoogle Scholar
  62. 62.
    Wilk T, Wozniak M (2012) Soft computing methods applied to combination of one-class classifiers. Neurocomputing 75(1):185–193CrossRefGoogle Scholar
  63. 63.
    Yeh CY, Lee ZY, Lee SJ (2009) Boosting one-class support vector machines for multi-class classification. Appl Artif Intell 23(4):297–315CrossRefGoogle Scholar
  64. 64.
    Yi J, Sheng Y, Xu C (2003) Neural network based uniformity profile control of linear chemical-mechanical planarization. IEEE Trans Semicond Manuf 16(4):609–620CrossRefGoogle Scholar
  65. 65.
    Ypma A, Duin RPW (1998) Support objects for domain approximation. In: Proceedings of International Conference on Artificial Neural Networks, Skovde, SwedenGoogle Scholar
  66. 66.
    Zhang T, Ramakrishnan R, Livny M (1996) BIRCH: an efficient data clustering method for very large databases. In: Proceedings of ACM SIGMOD Conference on Management of Data, Montreal, Canada, pp 103–114Google Scholar
  67. 67.
    Zhuang L, Dai H (2006) Parameter optimization of kernel-based on-class classifier on imbalance learning. J Comput 1(7):32–40CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London 2014

Authors and Affiliations

  1. 1.School of Industrial Management EngineeringKorea UniversitySeoulSouth Korea
  2. 2.Department of Industrial EngineeringSeoul National UniversitySeoulSouth Korea

Personalised recommendations