Structural and Multidisciplinary Optimization

, Volume 57, Issue 4, pp 1611–1623 | Cite as

Support Vector enhanced Kriging for metamodeling with noisy data

  • Liming Chen
  • Haobo Qiu
  • Chen Jiang
  • Mi Xiao
  • Liang Gao


Sample data may be corrupted by noise in engineering problems. In order to make satisfactory approximations for the data with noise, some regression metamodels are adopted in current researches. The commonly used nugget-effect Kriging regards the variance of noise as a constant and ignores the difference of the noise influence, thus may not be effective enough in some cases. Therefore, a Kriging-based metamodel which combines the merits of Kriging and Support Vector Regression (SVR) is put forward for improving the performance in metamodeling with noisy data. The developed method, termed as SVEK, can capture the underlying trend of an unknown function efficiently by classifying the sample points and then regressing these classified points with different extents. Besides, a criterion for selecting the error margin ε in SVR training is proposed to facilitate the parameter setting process. Moreover, a one-variable test example is used to illustrate the modeling theory and construction procedures of SVEK. Eight numerical benchmark problems with different important characteristics are used to validate the proposed method. Then an overall comparison between the nugget-effect Kriging and the proposed method has been made. Results show that SVEK is promising in metamodeling with noisy data.


Metamodeling Noisy observation Kriging Support Vector Regression (SVR) 



This work was supported in part by National Natural Science Foundation of China under Grant No 51675198, the 973 National Basic Research Program of China under Grant No 2014CB046705, and the National Natural Science Foundation of China under Grant No 51421062.


  1. Ankenman B, Nelson BL, Staum J (2010) Stochastic kriging for simulation metamodeling. Oper Res 58(2):371–382MathSciNetCrossRefzbMATHGoogle Scholar
  2. Binois M, Gramacy RB, Ludkovski M (2016) Practical heteroskedastic Gaussian process modeling for large simulation experiments. arXiv preprint arXiv:1611.05902Google Scholar
  3. Chang CC, Lin CJ (2011) LIBSVM: a library for support vector machines. ACM Trans Intell Syst Technol 2(3):27CrossRefGoogle Scholar
  4. Cherkassky V, Ma Y (2004) Practical selection of SVM parameters and noise estimation for SVM regression. Neural Netw 17(1):113–126CrossRefzbMATHGoogle Scholar
  5. Clarke SM, Griebsch JH, Simpson TW (2005) Analysis of support vector regression for approximation of complex engineering analyses. J Mech Des 127(6):1077–1087CrossRefGoogle Scholar
  6. Cressie NAC (1993) Statistics for Spatial Data, revised edition. Wiley, New YorkGoogle Scholar
  7. Dette H, Pepelyshev A (2010) Generalized Latin hypercube design for computer experiments. Technometrics 52(4):421–429MathSciNetCrossRefGoogle Scholar
  8. Dey S, Mukhopadhyay T, Sahu SK, Adhikari S (2016a) Effect of cutout on stochastic natural frequency of composite curved panels. Compos Part B 105:188–202CrossRefGoogle Scholar
  9. Dey S, Mukhopadhyay T, Spickenheuer A, Adhikari S, Heinrich G (2016b) Bottom up surrogate based approach for stochastic frequency response analysis of laminated composite plates. Compos Struct 140:712–727CrossRefGoogle Scholar
  10. Dey S, Naskar S, Mukhopadhyay T, Gohs U, Spickenheuer U, Bittrich L, Sriramula S, Adhikari S, Heinrich G (2016c) Uncertain natural frequency analysis of composite plates including effect of noise–A polynomial neural network approach. Compos Struct 143:130–142CrossRefGoogle Scholar
  11. Forrester AIJ, Sóbester A, Keane AJ (2008) Engineering design via surrogate modelling: a practical guide. Wiley, ChichesterGoogle Scholar
  12. Forrester AIJ, Keane AJ (2009) Recent advances in surrogate-based optimization. Prog Aerosp Sci 45(1):50–79CrossRefGoogle Scholar
  13. Forrester AIJ, Keane AJ, Bressloff NW (2006) Design and analysis of" Noisy" computer experiments. AIAA J 44(10):2331–2339CrossRefGoogle Scholar
  14. Gneiting T, Raftery AE (2007) Strictly proper scoring rules, prediction, and estimation. J Am Stat Assoc 102(477):359–378MathSciNetCrossRefzbMATHGoogle Scholar
  15. Goldberg PW, Williams CKI, Bishop CM (1998) Regression with input-dependent noise: A Gaussian process treatment. In: Advances in neural information processing systems. pp 493–499Google Scholar
  16. Gramacy RB, Lee HKH (2012) Cases for the nugget in modeling computer experiments. Stat Comput 22(3):713–722MathSciNetCrossRefzbMATHGoogle Scholar
  17. Huang D, Allen TT, Notz WI, Zeng N (2006) Global optimization of stochastic black-box systems via sequential kriging meta-models. J Glob Optim 34(3):441–466MathSciNetCrossRefzbMATHGoogle Scholar
  18. Jalali H, Van Nieuwenhuyse I, Picheny V (2017) Comparison of Kriging-based algorithms for simulation optimization with heterogeneous noise. Eur J Oper Res 261(1):279–301MathSciNetCrossRefGoogle Scholar
  19. Jones DR (2001) A taxonomy of global optimization methods based on response surfaces. J Glob Optim 21(4):345–383MathSciNetCrossRefzbMATHGoogle Scholar
  20. Keane A, Nair P (2005) Computational approaches for aerospace design: the pursuit of excellence. Wiley, ChichesterCrossRefGoogle Scholar
  21. Kersting K, Plagemann C, Pfaff P, Burgard W (2007) Most likely heteroscedastic Gaussian process regression. In: Proceedings of the 24th international conference on Machine learning. ACM, pp 393–400Google Scholar
  22. Kitayama S, Arakawa M, Yamazaki K (2011) Sequential approximate optimization using radial basis function network for engineering optimization. Optim Eng 12(4):535–557CrossRefzbMATHGoogle Scholar
  23. Kwok JT, Tsang IW (2003) Linear dependency between ε and the input noise in ε-support vector regression. IEEE Trans Neural Netw 14(3):544–553CrossRefGoogle Scholar
  24. Martin JD, Simpson TW (2005) Use of kriging models to approximate deterministic computer models. AIAA J 43(4):853–863CrossRefGoogle Scholar
  25. Matheron G (1963) Principles of geostatistics. Econ Geol 58(8):1246–1266CrossRefGoogle Scholar
  26. Matsumura T, Haftka RT, Kim NH (2015) Accurate predictions from noisy data: replication versus exploration with applications to structural failure. Struct Multidiscip Optim 51(1):23–40CrossRefGoogle Scholar
  27. Mattera D, Haykin S (1999) Support vector machines for dynamic reconstruction of a chaotic system. In: Schölkopf B., Burges C.J.C., and Smola A.J. (Eds.), Advances in Kernel Methods—Support Vector Learning. MIT Press, Cambridge, pp. 211–242.Google Scholar
  28. McKay MD, Beckman RJ, Conover WJ (1979) Comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics 21(2):239–245MathSciNetzbMATHGoogle Scholar
  29. Molga M, Smutnicki C (2005) Test functions for optimization needs. Test functions for optimization needs. Retrieved in June 2017, from
  30. Mukhopadhyay T, Naskar S, Dey S, Adhikari S (2016) On quantifying the effect of noise in surrogate based stochastic free vibration analysis of laminated composite shallow shells. Compos Struct 140:798–805CrossRefGoogle Scholar
  31. Mukhopadhyay T, Chakraborty S, Dey S, Adhikari S, Chowdhury R (2017) A critical assessment of kriging model variants for high-fidelity uncertainty quantification in dynamics of composite shells. Arch Comput Meth Eng 24(3):495–518MathSciNetCrossRefzbMATHGoogle Scholar
  32. Naskar S, Mukhopadhyay T, Sriramula S, Adhikari S (2017) Stochastic natural frequency analysis of damaged thin-walled laminated composite beams with uncertainty in micromechanical properties. Compos Struct 160:312–334CrossRefGoogle Scholar
  33. Papila M, Haftka RT (2000) Response surface approximations: noise, error repair, and modeling errors. AIAA J 38(12):2336–2343CrossRefGoogle Scholar
  34. Picheny V, Ginsbourger D (2014) Noisy kriging-based optimization methods: a unified implementation within the DiceOptim package. Comput Stat Data Anal 71:1035–1053MathSciNetCrossRefGoogle Scholar
  35. Picheny V, Wagner T, and Ginsbourger D (2013) A benchmark of kriging-based infill criteria for noisy optimization. Structural and multidisciplinary optimization 48.3:607–626Google Scholar
  36. Queipo NV, Haftka RT, Shyy W, Goel T (2005) Surrogate-based analysis and optimization. Prog Aerosp Sci 41(1):1–28CrossRefGoogle Scholar
  37. Sacks J, Welch WJ, Mitchell TJ, Wynn HP (1989) Design and analysis of computer experiments. Stat Sci 4:409–423MathSciNetCrossRefzbMATHGoogle Scholar
  38. Sakata S, Ashida F (2009) Ns-kriging based microstructural optimization applied to minimizing stochastic variation of homogenized elasticity of fiber reinforced composites. Struct Multidiscip Optim 38(5):443–453CrossRefGoogle Scholar
  39. Sakata S, Ashida F, Zako M (2008) Microstructural design of composite materials using fixed-grid modeling and noise-resistant smoothed Kriging-based approximate optimization. Struct Multidiscip Optim 36(3):273–287CrossRefGoogle Scholar
  40. Schölkopf B, Smola AJ (2002) Learning with kernels: support vector machines, regularization, optimization, and beyond. MIT press, CambridgeGoogle Scholar
  41. Smola AJ, Schölkopf B (2004) A tutorial on support vector regression. Stat Comput 14(3):199–222MathSciNetCrossRefGoogle Scholar
  42. Soares C, Brazdil PB, Kuba P (2004) A meta-learning method to select the kernel width in support vector regression. Mach Learn 54(3):195–209CrossRefzbMATHGoogle Scholar
  43. Üstün B, Melssen WJ, Oudenhuijzen M, Buydens LMC (2005) Determination of optimal support vector regression parameters by genetic algorithms and simplex optimization. Anal Chim Acta 544(1):292–305CrossRefGoogle Scholar
  44. Wang GG, Shan S (2007) Review of metamodeling techniques in support of engineering design optimization. J Mech Des 129(4):370–380CrossRefGoogle Scholar
  45. Wang H, Shan S, Wang GG (2011) Integrating least square support vector regression and mode pursuing sampling optimization for crashworthiness design. J Mech Des 133(4):041002CrossRefGoogle Scholar
  46. Wiebenga JH, Van den Boogaard AH (2014) On the effect of numerical noise in approximate optimization of forming processes using numerical simulations. Int J Mater Form 7(3):317–335Google Scholar
  47. Yin J, Ng SH, Ng KM (2011) Kriging metamodel with modified nugget-effect: The heteroscedastic variance case. Comput Ind Eng 61(3):760–777MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany 2017

Authors and Affiliations

  • Liming Chen
    • 1
  • Haobo Qiu
    • 1
  • Chen Jiang
    • 1
  • Mi Xiao
    • 1
  • Liang Gao
    • 1
  1. 1.State Key Laboratory of Digital Manufacturing Equipment and Technology, School of Mechanical Science and EngineeringHuazhong University of Science & TechnologyWuhanPeople’s Republic of China

Personalised recommendations