Abstract
Multi-fidelity surrogate modelling offers an efficient way to approximate computationally expensive simulations. In particular, Kriging-based surrogate models are popular for approximating deterministic data. In this work, the performance of Kriging is investigated when multi-fidelity gradient data is introduced along with multi-fidelity function data to approximate computationally expensive black-box simulations. To achieve this, the recursive CoKriging formulation is extended by incorporating multi-fidelity gradient information. This approach, denoted by Gradient-Enhanced recursive CoKriging (GECoK), is initially applied to two analytical problems. As expected, results from the analytical benchmark problems show that additional gradient information of different fidelities can significantly improve the accuracy of the Kriging model. Moreover, GECoK provides a better approximation even when the gradient information is only partially available. Further comparison between CoKriging, Gradient Enhanced Kriging, denoted by GEK, and GECoK highlights various advantages of employing single and multi-fidelity gradient data. Finally, GECoK is further applied to two real-life examples.
Similar content being viewed by others
Notes
www.eesof.com, Agilent Technologies EEsof EDA, Santa Rosa, CA.
www.cst.com, CST Computer Simulation Technology AG, Darmstadt, Germany.
References
Bandler JW, Biernacki RM, Chen SH, Grobelny PA, Hemmers RH (1994) Space mapping technique for electromagnetic optimization. IEEE Trans Microw Theory Tech 12:2536–2544
Brezillon J, Dwight R (2005) Discrete adjoint of the Navier-stokes Equations for aerodynamic shape optimization. In: Evolutionary and deterministic methods for design, optimisation and control with applications to industrial and societal problems (EUROGEN). Munich, Germany, p 2005
Chemmangat K, Ferranti F, Knockaert L, Dhaene T (2011) Parametric macromodeling for sensitivity responses from tabulated data. IEEE Microw Wirel Components Lett 21(8):397–399
Chung HS, Alonso JJ (2002) Using gradients to construct Cokriging approximation models for high-dimensional design optimization problems. In: Problems, 40th AIAA aerospace sciences meeting and exhibit, AIAA. Reno, NV, pp 2002–0317
Couckuyt I, Declercq F, Dhaene T, Rogier H, Knockaert L (2010) Surrogate-based infill optimization applied to electromagnetic problems. Int J RF Microw Comput-Aided Eng 20(5):492–501
Courrier N, Boucard PA, Soulier B (2014) The use of partially converged simulations in building surrogate models. Adv Eng Softw 67(0):186–197
Craig PS, Goldstein M, Seheult AH, Smith JA (1998) Constructing partial prior specifications for models of complex physical systems. J R Stat Soc Ser D : (Stat) 47(1):37–53
Cumming JA, Goldstein M (2009) Small sample Bayesian designs for complex high-dimensional models based on information gained using fast approximations. Technometrics 51(4):377–388
Davis GJ, Morris MD (1997) Six factors which affect the condition number of matrices associated with kriging. Math Geol 29(5):669–683
Degroote J, Hojjat M, Stavropoulou E, Wüchner R, Bletzinger KU (2013) Partitioned solution of an unsteady adjoint for strongly coupled fluid-structure interactions and application to parameter identification of a one-dimensional problem. Struct Multidiscip Optim 47(1):77–94
Dwight RP, Han ZH (2009) Efficient uncertainty quantification using gradient-enhanced kriging. In: 11th AIAA on-deterministic approaches conference. Palm Springs, California
Forrester AI, Bressloff NW, Keane AJ (2006) Optimization using surrogate models and partially converged computational fluid dynamics simulations. Proc R Soc A : Math Phys Eng Sci Med 462(2071):2177–2204
Forrester AI, Sóbester A, Keane AJ (2007) Multi-fidelity optimization via surrogate modelling. Proc R Soc 463:3251–3269
Forrester AI, Sóbester A, Keane AJ (2008) Engineering design via surrogate modelling: a practical guide, 1st edn. Wiley, New York
Goldstein M, Wooff DA (2007) Bayes linear statistics: theory & methods bayes linear statistics. Wiley, New York
Higdon D, Kennedy M, Cavendish J, Cafeo J, Ryne RD (2004) Combining field data and computer simulations for calibration and prediction. SIAM J Sci Comput 26(2):448–466
Huang D, Allen T, Notz W, Miller R (2006) Sequential kriging optimization using multiple-fidelity evaluations. Struct Multidiscip Optim 32(5):369–382
Jin R, Chen W, Simpson TW (2000) Comparative studies of metamodeling techniques under multiple modeling criteria. Struct Multidiscip Optim 23:1–13
Kennedy MC, O’Hagan A (2000) Predicting the output from a complex computer code when fast approximations are available. Biometrika 87(1):1–13
Kleijnen J (2009) Kriging metamodeling in simulation: A review. Eur J Oper Res 192(3):707–716
Laurenceau J, Sagaut P (2008) Building efficient response surfaces of aerodynamic functions with kriging and cokriging. AIAA 46(2):498–507
Laurenceau J, Meaux M, Montagnac M, Sagaut P (2010) Comparison of gradient-based and gradient-enhanced response-surface-based optimizers. Am Inst Aeronaut Astronaut J 48(5):981–994
Laurent L, Boucard P A, Soulier B (2013) Generation of a cokriging metamodel using a multiparametric strategy. Comput Mech 51(2):151–169
Le Gratiet L (2012) Recursive co-kriging model for design of computer experiments with multiple levels of fidelity with an application to hydrodynamic. arXiv:http://arxiv.org/abs/1210.0686
Leary SJ, Bhaskar A, Keane AJ (2004) A derivative based surrogate model for approximating and optimizing the output of an expensive computer simulation. J Glob Optim 30(1):39–58
Liu W (2003) Development of gradient-enhanced kriging approximations for multidisciplinary design optimisation. PhD thesis, University of Notre Dame, Notre Dame, Indiana
March A, Willcox K, Wang Q (2010) Gradient-based multifidelity optimisation for aircraft design using bayesian model calibration. In: 2nd aircraft structural design conference. Royal Aeronautical Society, London, p 1720
Morris MD, Mitchell TJ, Ylvisaker D (1993) Bayesian design and analysis of computer experiments: use of gradients in surface prediction. Technometrics 35(3):243–255
Näther W, Šimák J (2003) Effective observation of random processes using derivatives. Metrika 58:71–84
Qian PZG, Wu CFJ (2008) Bayesian hierarchical modeling for integrating low-accuracy and high-accuracy experiments. Technometrics 50(2):192–204
Rasmussen CE, Williams CKI (2006) Gaussian processes for machine learning. The MIT Press. MA, USA
Sacks J, Schiller SB, Welch WJ (1989) Designs for computer experiments. Technometrics 31(1):41–47
Sacks J, Welch WJ, Mitchell TJ, Wynn HP (1989b) Design and analysis of computer experiments. Stat Sci 4(4):409–423
Schneider R (2012) FEINS: finite element solver for shape optimization with adjoint equations. In: Progress in industrial mathematics at ECMI 2010 conference, pp 573–580
Simpson T, Poplinski J, Koch PN, Allen J (2001) Metamodels for computer-based engineering design: Survey and recommendations. Engineering with Computers 17(2):129–150
Stein ML (1999) Interpolation of spatial data: some theory for kriging. Springer, New York
Toal DJ, Forrester AI, Bressloff NW, Keane AJ, Holden C (2009) An adjoint for likelihood maximization. Proc R Soc A 8(465):3267–3287
Šimák J (2002) On experimental designs for derivative random fields. PhD thesis. TU Bergakademie Freiberg. Freiberg, Germany
Wang GG, Shan S (2006) Review of metamodeling techniques in support of engineering design optimization. J Mech Des 129(4):370–380
Yu W, Bandler J (2006) Optimization of spiral inductor on silicon using space mapping. In: IEEE MTT-S international microwave symposium digest, pp 1085–1088
Zimmermann R (2010) Asymptotic behavior of the likelihood function of covariance matrices of spatial gaussian processes. J Appl Math. http://www.hindawi.com/journals/jam/2010/494070/cta/
Zimmermann R (2013) On the maximum likelihood training of gradient-enhanced spatial gaussian processes. SIAM J Sci Comput 35(6):A2554—A2574
Acknowledgments
This research has been funded by the Interuniversity Attraction Poles Programme BESTCOM initiated by the Belgian Science Policy Office. Additionally, this research has been supported by the Fund for Scientific Research in Flanders (FWO-Vlaanderen). Ivo Couckuyt and Francesco Ferranti are post-doctoral research fellows of the Research Foundation Flanders (FWO-Vlaanderen). The authors like to thank Frank Mosler from Computer Simulation Technology (CST) for providing the microwave inter-digital filter example.
Author information
Authors and Affiliations
Corresponding author
Appendices
Appendix A: Analytical expressions for gradient, Hessian and likelihood gradients of correlation functions
1.1 A.1 Gaussian correlation function:
Gradient of correlation function with respect to X (i.e., cross-correlation):
Hessian of correlation function with respect to X (i.e., cross-correlation):
Derivative of correlation function with respect to θ k :
Derivatives of cross-correlation functions with respect to θ k :
1.2 A.2 Matérn \(\frac {5}{2}\) correlation function:
Gradient of correlation function with respect to X (i.e., cross-correlation):
Hessian of correlation function with respect to X (i.e., cross-correlation):
Derivative of correlation function with respect to θ k :
Derivatives of cross-correlation functions with respect to θ k :
where
Appendix B: Comparison of MLE and Least Squares Estimation (LSE) of scaling parameter (ρ)
Rights and permissions
About this article
Cite this article
Ulaganathan, S., Couckuyt, I., Ferranti, F. et al. Performance study of multi-fidelity gradient enhanced kriging. Struct Multidisc Optim 51, 1017–1033 (2015). https://doi.org/10.1007/s00158-014-1192-x
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00158-014-1192-x