Skip to main content
SpringerLink
Log in

A regularization method for constructing trend function in Kriging model

  • Research Paper
  • Published:
Structural and Multidisciplinary Optimization Aims and scope Submit manuscript

Abstract

Kriging is a popular surrogate for approximating computationally expensive computer experiments. When sample points are limited, it is difficult to identify the overall trend of the problem at hand properly. Thanks to the interpolating characteristic of the Kriging model, a constant is widely used as the trend function, which neglects the overall trend presented by data. However, previous researches prove that an appropriate trend function considering high-order terms is able to enhance the approximation ability of the Kriging model. In this paper, a regularization approach is proposed to construct the trend function in the Kriging model to improve the prediction accuracy. First, a new weighting scheme, which is formulated as an optimization problem with regularization terms, is used to solve the regression coefficients. Then, the other model parameters are estimated by maximizing the likelihood function, which leads to a nested optimization problem. It is solved iteratively to obtain the final estimation of the model parameters. From a Bayesian point of view, the proposed regularization method can adaptively tune the parameter of the prior distribution on the regression coefficients in the iterative algorithm. To select good regularization parameters, a cross-validation method is used. The implementation is tested on several analytical examples and physical examples, and the experimental results confirm the effectiveness of the proposed approach.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  • Acar E (2013) Effects of the correlation model, the trend model, and the number of training points on the accuracy of kriging metamodels. Expert Syst 30(5):418–428

    Article  Google Scholar 

  • Acar E (2014) Simultaneous optimization of shape parameters and weight factors in ensemble of radial basis functions. Struct Multidiscip Optim 49(6):969–978

    Article  Google Scholar 

  • Aute V, Saleh K, Abdelaziz O, Azarm S, Radermacher R (2013) Cross-validation based single response adaptive design of experiments for Kriging metamodeling of deterministic computer simulations. Struct Multidiscip Optim 48(3):581–605

    Article  Google Scholar 

  • Balabanov VO, Weckner O, Wu J (2014) Reducing error of polynomial approximation outside of designated design space for practical problems. In: 15th AIAA/ISSMO multidisciplinary analysis and optimization conference, p 2303

  • Ben-Ari EN, Steinberg DM (2007) Modeling data from computer experiments: sn empirical comparison of Kriging with mars and projection pursuit regression. Qual Eng 19(4):327–338

    Article  Google Scholar 

  • Bouhlel MA, Bartoli N, Otsmane A, Morlier J (2016) Improving kriging surrogates of high-dimensional design models by partial least squares dimension reduction. Struct Multidiscip Optim 53(5):935–952

    Article  MathSciNet  MATH  Google Scholar 

  • Clark LD, Bae H, Gobal K, Penmetsa RC (2016) Engineering design exploration using locally optimized covariance Kriging. AIAA J 54(10):3160–3175

    Article  Google Scholar 

  • Clarke SM, Griebsch JH, Simpson TW (2005) Analysis of support vector regression for approximation of complex engineering analyses. J Mech Des 127(6):1077–1087

    Article  Google Scholar 

  • Dette H, Pepelyshev A (2010) Generalized latin hypercube design for computer experiments. Technometrics 52(4):421–429

    Article  MathSciNet  Google Scholar 

  • Echard B, Gayton N, Lemaire M (2011) Ak-mcs: an active learning reliability method combining Kriging and monte carlo simulation. Struct Saf 33(2):145–154

    Article  Google Scholar 

  • Forrester AIJ, Keane AJ (2009) Recent advances in surrogate-based optimization. Progress Aerosp Sci 45(1):50–79

    Article  Google Scholar 

  • Forrester AIJ, Sóbester A, Keane AJ (2008) Engineering design via surrogate modelling: a practical guide. Wiley

  • Friedman JH (1991) Multivariate adaptive regression splines. Ann Statist 19(1):1–67

    Article  MathSciNet  MATH  Google Scholar 

  • Galatsanos NP, Katsaggelos AK (1992) Methods for choosing the regularization parameter and estimating the noise variance in image restoration and their relation. IEEE Trans Image Process 1(3):322–336

    Article  Google Scholar 

  • Ginsbourger D, Dupuy D, Badea A, Carraro L, Roustant O (2010) A note on the choice and the estimation of Kriging models for the analysis of computer experiments. Appl Stoch Model Bus Ind 25(2):115–131

    Article  MathSciNet  MATH  Google Scholar 

  • Goel T, Hafkta RT, Shyy W (2009) Comparing error estimation measures for polynomial and Kriging approximation of noise-free functions. Struct Multidiscip Optim 38(5):429–442

    Article  Google Scholar 

  • Golub GH, Heath M, Wahba G (1979) Generalized cross-validation as a method for choosing a good ridge parameter. Technometrics 21(2):215–223

    Article  MathSciNet  MATH  Google Scholar 

  • Hoerl AE, Kennard RW (1970) Ridge regression: biased estimation for nonorthogonal problems. Technometrics 12(1):55–67

    Article  MATH  Google Scholar 

  • Hung Y (2011) Penalized blind Kriging in computer experiments. Stat Sin 21(3):1171–1190

    Article  MathSciNet  MATH  Google Scholar 

  • Joseph VR, Hung Y, Sudjianto A (2008) Blind Kriging: a new method for developing metamodels. J Mech Des 130(3):1–8

    Article  Google Scholar 

  • Kersaudy P, Sudret B, Varsier N, Picon O, Wiart J (2015) A new surrogate modeling technique combining Kriging and polynomial chaos expansions–application to uncertainty analysis in computational dosimetry. J Comput Phys 286:103–117

    Article  MathSciNet  MATH  Google Scholar 

  • Kwon H, Choi S (2015) A trended kriging model with r2 indicator and application to design optimization. Aerosp Sci Technol 43:111–125

    Article  Google Scholar 

  • Lee H, Lee DJ, Kwon H (2018) Development of an optimized trend Kriging model using regression analysis and selection process for optimal subset of basis functions. Aerosp Sci Technol 77:273–285

    Article  Google Scholar 

  • Li R, Sudjianto A (2005) Analysis of computer experiments using penalized likelihood in gaussian Kriging models. Technometrics 47(2):111–120

    Article  MathSciNet  Google Scholar 

  • Liang H, Zhu M (2013) Comment on “metamodeling method using dynamic Kriging for design optimization”. AIAA J 51(12):2988–2989

    Article  Google Scholar 

  • Liang H, Zhu M, Wu Z (2014) Using cross-validation to design trend function in Kriging surrogate modeling. AIAA J 52(10):2313–2327

    Article  Google Scholar 

  • Liu H, Ong YS, Cai J (2018) A survey of adaptive sampling for global metamodeling in support of simulation-based complex engineering design. Struct Multidiscip Optim 57(1):393–416

    Article  Google Scholar 

  • Lophaven SN, Nielsen HB, Søndergaard J (2002) Dace–a matlab kriging toolbox–version 2.0. Tech. rep

  • Martin JD, Simpson TW (2005) Use of kriging models to approximate deterministic computer models. AIAA J 43(4):853–863

    Article  Google Scholar 

  • McIlhagga W (2016) Penalized: a matlab toolbox for fitting generalized linear models with penalties. J Statist Softw 72(6):1–21

    Article  Google Scholar 

  • Mehmani A, Chowdhury S, Messac A (2015) Predictive quantification of surrogate model fidelity based on modal variations with sample density. Struct Multidiscip Optim 52(2):353–373

    Article  Google Scholar 

  • Park MY, Hastie T (2007) L1-regularization path algorithm for generalized linear models. J R Stat Soc 69(4):659–677

    Article  MathSciNet  Google Scholar 

  • Parmar M, Haselbacher A, Balachandar S (2010) Improved drag correlation for spheres and application to shock-tube experiments. AIAA J 48(6):1273–1277

    Article  Google Scholar 

  • Sacks J, Welch WJ, Mitchell TJ, Wynn HP (1989) Design and analysis of computer experiments. Stat Sci 4(4):409–423

    Article  MathSciNet  MATH  Google Scholar 

  • Schobi R, Sudret B, Wiart J (2015) Polynomial-chaos-based Kriging. Int J Uncertain Quantif 5(2):171–193

    Article  MathSciNet  Google Scholar 

  • Sóbester A, Forrester AI, Toal DJ, Tresidder E, Tucker S (2014) Engineering design applications of surrogate-assisted optimization techniques. Optim Eng 15(1):243–265

    Article  MATH  Google Scholar 

  • Tibshirani R (1996) Regression shrinkage and selection via the lasso. J R Stat Soc 58(1):267–288

    MathSciNet  MATH  Google Scholar 

  • Viana FAC (2011) Surrogates toolbox user’s guide. Gainesville, FL, Version 3.0. https://sites.google.com/site/srgtstoolbox/

  • Viana FAC, Haftka RT, Steffen V (2009) Multiple surrogates: how cross-validation errors can help us to obtain the best predictor. Struct Multidiscip Optim 39(4):439–457

    Article  Google Scholar 

  • Viana FAC, Simpson TW, Balabanov V, Toropov V (2014) Metamodeling in multidisciplinary design optimization: how far have we really come? AIAA J 52(4):670–690

    Article  Google Scholar 

  • Wang GG, Shan S (2007) Review of metamodeling techniques in support of engineering design optimization. J Mech Des 129(4):370–380

    Article  Google Scholar 

  • Yao W, Chen X, Ouyang Q, Tooren MV (2012a) A surrogate based multistage-multilevel optimization procedure for multidisciplinary design optimization. Struct Multidiscip Optim 45(4):559– 574

    Article  Google Scholar 

  • Yao W, Chen X, Zhao Y, Tooren MV (2012b) Concurrent subspace width optimization method for rbf neural network modeling. IEEE Trans Neural Netw Learn Syst 23(2):247–259

    Article  Google Scholar 

  • Yao W, Chen X, Ouyang Q, Tooren MV (2013) A reliability-based multidisciplinary design optimization procedure based on combined probability and evidence theory. Struct Multidiscip Optim 48(2):339–354

    Article  MathSciNet  Google Scholar 

  • Zhang S, Zhu P, Chen W, Arendt P (2013) Concurrent treatment of parametric uncertainty and metamodeling uncertainty in robust design. Struct Multidiscip Optim 47(1):63–76

    Article  MathSciNet  MATH  Google Scholar 

  • Zhang Y, Kim NH, Park C, Haftka RT (2016) Function extrapolation of noisy data using converging lines. In: AIAA Modeling and simulation technologies conference, p 2144

  • Zhang Y, Park C, Kim NH, Haftka RT (2017) Function prediction at one inaccessible point using converging lines. J Mech Des 139(5):051,402

    Article  Google Scholar 

  • Zhao L, Choi K, Lee I (2011) Metamodeling method using dynamic Kriging for design optimization. AIAA J 49(9):2034–2046

    Article  Google Scholar 

  • Zhao L, Choi K, Lee I (2013) Reply by the authors to the comment by H. Liang and M. Zhu. AIAA J 51(12):2989–2990

    Article  Google Scholar 

  • Zou H (2006) The adaptive lasso and its oracle properties. J Am Stat Assoc 101:1418–1429

    Article  MathSciNet  MATH  Google Scholar 

  • Zou H, Hastie T (2005) Regularization and variable selection via the elastic net. J R Stat Soc 67(2):301–320

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

We would like to thank four reviewers for thoughtful comments that lead to improvements in the manuscript.

Funding

This work is supported by the National Natural Science Foundation of China (No. 11725211 and 51675525).

Author information

Authors and Affiliations

  1. College of Aerospace Science and Engineering, National University of Defense Technology, Changsha, 410073, Hunan, China

    Yi Zhang & Siyu Ye

  2. National Innovation Institute of Defense Technology, Chinese Academy of Military Science, Beijing, 100000, China

    Wen Yao & Xiaoqian Chen

Authors
  1. Yi Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wen Yao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Siyu Ye
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiaoqian Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wen Yao.

Additional information

Responsible Editor: Raphael Haftka

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

The codes can be downloaded from here: https://github.com/zhangyi413/PK.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, Y., Yao, W., Ye, S. et al. A regularization method for constructing trend function in Kriging model. Struct Multidisc Optim 59, 1221–1239 (2019). https://doi.org/10.1007/s00158-018-2127-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00158-018-2127-8

Keywords

Search

Navigation