Skip to main content

Variable-fidelity expected improvement method for efficient global optimization of expensive functions

Abstract

The efficient global optimization method (EGO) based on kriging surrogate model and expected improvement (EI) has received much attention for optimization of high-fidelity, expensive functions. However, when the standard EI method is directly applied to a variable-fidelity optimization (VFO) introducing assistance from cheap, low-fidelity functions via hierarchical kriging (HK) or cokriging, only high-fidelity samples can be chosen to update the variable-fidelity surrogate model. The theory of infilling low-fidelity samples towards the improvement of high-fidelity function is still a blank area. This article proposes a variable-fidelity EI (VF-EI) method that can adaptively select new samples of both low and high fidelity. Based on the theory of HK model, the EI of the high-fidelity function associated with adding low- and high-fidelity sample points are analytically derived, and the resulting VF-EI is a function of both the design variables x and the fidelity level l. Through maximizing the VF-EI, both the sample location and fidelity level of next numerical evaluation are determined, which in turn drives the optimization converging to the global optimum of high-fidelity function. The proposed VF-EI is verified by six analytical test cases and demonstrated by two engineering problems, including aerodynamic shape optimizations of RAE 2822 airfoil and ONERA M6 wing. The results show that it can remarkably improve the optimization efficiency and compares favorably to the existing methods.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

References

  • Ackley DH (1987) A connectionist machine for genetic hill-climbing. Kluwer, Boston

    Book  Google Scholar 

  • Alexandrov N, Dennis JE, Lewis RM, Torczon V (1998) A trust-region framework for managing the use of approximation models in optimization. Struct Optim 15(1):16–23

    Article  Google Scholar 

  • Alexandrov N, Lewis RM, Gumbert CR, Green LL, Newman PA (2001) Approximation and model management in aerodynamic optimization with variable-fidelity models. J Aircr 38(6):1093–1101

    Article  Google Scholar 

  • Bakr MH, Bandler JW, Madsen K, SØndergaard J (2001) An introduction to the space mapping technique. Optim Eng 2(4):369–384

    Article  MATH  MathSciNet  Google Scholar 

  • Benamara T, Breitkopt P, Lepot I, Sainvitu C (2016) Adaptive infill sampling criterion for multi-fidelity optimization based on Gappy-POD. Struct Multidisc Optim 54(4):843–855

    Article  Google Scholar 

  • Cai X, Qiu H, Gao L, Wei L, Shao X (2017) Adaptive radial-basis-function-based multifidelity metamodeling for expensive black-box problems. AIAA J 55(7):2424–2436

    Article  Google Scholar 

  • Chang KJ, Haftka RT, Giles GL, Kao PJ (1993) Sensitivity-based scaling for approximation structural response. J Aircr 30(2):283–288

    Article  Google Scholar 

  • Choi S, Alonso JJ, Kroo IM, Wintzer M (2004) Multi-fidelity design optimization of low-boom supersonic business jets. In: 10th AIAA/ISSMO Multidiscip Anal Optim Conf, AIAA paper 2004–4371, Albany, NY, US, 30 August-1 September

  • Choi S, Alonso JJ, Kim S, Kroo IM (2009) Two-level multifidelity design optimization studies for supersonic jets. J Aircr 46(3):776–790

    Article  Google Scholar 

  • Courrier N, Boucard PA, Soulier B (2016) Variable-fidelity modeling of structural analysis of assemblies. J Glob Optim 64(3):577–613

    Article  MATH  MathSciNet  Google Scholar 

  • Forrester AIJ, Keane AJ (2009) Recent advances in surrogate-based optimization. Prog Aerosp Sci 45(1–3):50–79

    Article  Google Scholar 

  • Forrester AIJ, Sóbester A, Keane AJ (2007) Multi-fidelity optimization via surrogate modelling. In: Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences 463(2088):3251–3269

  • Forrester AIJ, Sóbester A, Keane AJ (2008) Engineering design via surrogate modelling— a practical guide. Wiley, New York

    Book  Google Scholar 

  • Gano SE, Renaud JE, Sanders B (2005) Hybrid variable fidelity optimization by using a kriging-based scaling function. AIAA J 43(11):2422–2430

    Article  Google Scholar 

  • Giunta AA, Wojtkiewicz SF, Eldred MS (2003) Overview of modern design of experiments methods for computational simulations. In: 41st Aeros Sci Meet Exhib, AIAA paper 2003–649, Reno, Nevada, 6–9 January

  • Ha H, Oh S, Yee K (2014) Feasibility study of hierarchical kriging model in the design optimization process. J Korean Soc Aeronaut Space Sci 42(2):108–118

    Google Scholar 

  • Haftka RT (1991) Combining global and local approximations. AIAA J 29(9):1523–1525

    Article  Google Scholar 

  • Han Z-H (2016a) SurroOpt: a generic surrogate-based optimization code for aerodynamic and multidisciplinary design. In: 30th Cong. Int. Counc. Aeronaut. Sci. ICAS, paper no. 2016–0281, Daejeon, Korea, 25–30 September

  • Han Z-H (2016b) Kriging surrogate model and its application to design optimization: A review of recent progress. Chin J Aeronaut 37(11):3197–3225

    Google Scholar 

  • Han Z-H, Görtz S (2012) Hierarchical kriging model for variable-fidelity surrogate modeling. AIAA J 50(5):1285–1296

  • Han Z-H, Zhang K-S (2012) Surrogate-based optimization. In: Roeva O (ed) Real-World Applications of Genetic Algorithms, InTech, pp. 343–362

  • Han Z-H, He F, Song W-P, Qiao Z-D (2007) A preconditioned multigrid method for efficient simulation of three-dimensional compressible and incompressible flows. Chin J Aeronaut 20(4):289–296

    Article  Google Scholar 

  • Han Z-H, Zimmermann R, Görtz S (2012) An alternative cokriging model for variable-fidelity surrogate modeling. AIAA J 50(5):1205–1210

  • Han Z-H, Görtz S, Zimmermann R (2013) Improving variable-fidelity surrogate modeling via gradient-enhanced kriging and a generalized hybrid bridge function. Aerosp Sci Technol 25:177–189

  • Han Z-H, Zhang Y, Song C-X, Zhang K-S (2017) Weighted gradient-enhanced kriging for high-dimensional surrogate modeling and design optimization. AIAA J 55(12):4330–4346. https://doi.org/10.2514/1.J055842

    Article  Google Scholar 

  • Hartman JK (1973) Some experiments in global optimization. Nav Res Logist Q 20:569–576

    Article  MATH  Google Scholar 

  • Holland JH (1975) Adaptation in natural and artificial systems. Control & Artificial Intelligence University of Michigan Press, 6(2):126–137

  • Huang D, Allen TT, Notz WI, Miller RA (2006) Sequential kriging optimization using multi-fidelity evaluations. Struct Multidiscip Optim 32(5):369–382

    Article  Google Scholar 

  • Jo Y, Yi S, Choi S, Lee DJ, Choi DZ (2016) Adaptive variable-fidelity analysis and design using dynamic fidelity indicators. AIAA J 54(11):3564–3579

    Article  Google Scholar 

  • Jones DR (2001) A taxonomy of global optimization methods based on response surfaces. J Glob Optim 21(4):345–383

    Article  MATH  MathSciNet  Google Scholar 

  • Jones DR, Schonlau M, Welch WJ (1998) Efficient global optimization of expensive black-box functions. J Glob Optim 13(4):455–492

    Article  MATH  MathSciNet  Google Scholar 

  • Journel AG, Huijbregts CJ (1978) Mining geostatistics. Academic Press, New York

    Google Scholar 

  • Kennedy J, Eberhart R (1995) Particle swarm optimization. In: Proceedings of IEEE International Conference on Neural Networks. 4:1942–1948

  • Kennedy MC, O’Hagan A (2000) Predicting the output from a complex computer code when fast approximations are available. Biometrika 87(1):1–13

    Article  MATH  MathSciNet  Google Scholar 

  • Kim Y, Lee S, Yee K, Rhee D (2017) High-to-low initial sample ratio of hierarchical kriging for film hole array optimization. J Propuls Power. https://doi.org/10.2514/1.B36556

  • Koch PN, Simpson TW, Allen JK, Mistree F (1999) Statistical approximations for multidisciplinary design optimization: The problem of the size. J Aircr 36(1):275–286

    Article  Google Scholar 

  • Koziel S, Leifsson L (2013) Surrogate-based aerodynamic shape optimization by variable-resolution models. AIAA J 51(1):94–106

    Article  Google Scholar 

  • Koziel S, Leifsson L, Yang XS (2013) Surrogate-based optimization. In: Koziel S, Yang XS, Zhang QJ (eds) Simulation-driven design optimization and modeling for microwave engineering. Imperial College Press, London

    Chapter  Google Scholar 

  • Krige DG (1951) A statistical approach to some basic mine valuation problems on the Witwatersrand. J South Afr Inst Min Metall 52(6):119–139

    Google Scholar 

  • Kulfan BM (2008) Universal parametric geometry representation method. J Aircr 45(1):142–158

    Article  Google Scholar 

  • Leifsson L, Koziel S, Tesfahunegn YA (2016) Multiobjective aerodynamic optimization by variable-fidelity models and response surface surrogates. AIAA J 54(2):531–541

    Article  Google Scholar 

  • Liu J, Han Z-H., Song W-P (2012) Comparison of infill sampling criteria in kriging-based aerodynamic optimization. In: 28th Cong Int Counc Aeronaut Sci ICAS, Brisbane, Australia, 23–38 September

  • Liu J, Song W-P, Han Z-H, Zhang Y (2017) Efficient aerodynamic shape optimization of transonic wings using a parallel infilling strategy and surrogate models. Struct Multidiscip Optim 55(3):925–943

    Article  Google Scholar 

  • Martin JD, Simpson TW (2005) Use of kriging models to approximate deterministic computer models. AIAA J 43(4):853–863

    Article  Google Scholar 

  • Matheron GM (1963) Principles of geostatistics. Econ Geol 58(8):1246–1266

    Article  Google Scholar 

  • McDaniel WR, Ankenman BE (2000) A response surface test bed. Qual Relib Eng Int 16:363–372

    Article  Google Scholar 

  • Palar PS, Shimoyama K (2017) Multi-fidelity uncertainty analysis in CFD using hierarchical kriging. In: 35th AIAA Appl Aerodyn Conf, AIAA paper 2017–3261, Denver, Colorado, US, 5–9 June

  • Park C, Haftka RT, Kim NH (2017) Remarks on multi-fidelity surrogates. Struct Multidisc Optim 55(3):1029–1050

    Article  MathSciNet  Google Scholar 

  • Queipo NV, Haftka RT, Shyy W, Goela T, Vaidyanathan R, Tucker PK (2005) Surrogate-based analysis and optimization. Prog Aerosp Sci 45(1):1–28

    Article  Google Scholar 

  • Reisenthel PH, Allen TT (2014) Application of multifidelity expected improvement algorithms to aeroelastic design optimization. In: 10th AIAA Multidisc Des Optim Spec Conf, AIAA paper 2016–1542, San Diego, US, 7–10 January

  • Robinson TD, Eldred MS, Willcox KE, Haimes R (2008) Surrogate-based optimization using multifidelity models with variable parameterization and corrected space mapping. AIAA J 46(11):2814–2822

    Article  Google Scholar 

  • Sacks J, Welch WJ, Mitchell TJ, Wynn HP (1989) Design and analysis of computer experiments. Stat Sci 4(4):409–423

    Article  MATH  MathSciNet  Google Scholar 

  • Shan S, Wang GG (2010) Survey of modeling and optimization strategies to solve high-dimensional design problems with computationally-expensive black-box functions. Struct Multidiscip Optim 41(2):219–241

    Article  MATH  MathSciNet  Google Scholar 

  • Simpson TW, Mauery TM, Korte JJ, Mistree F (2001) Kriging models for global approximation in simulation-based multidisciplinary design optimization. AIAA J 39(12):2233–2241

    Article  Google Scholar 

  • Toal DJJ, Bressloff NW, Kean AJ (2008) Kriging hyperparameter tuning strategies. AIAA J 46(5):1240–1252

    Article  Google Scholar 

  • Viana FAC, Simpson TW, Balabanov V, Toropov V (2014) Metamodeling in multidisciplinary design optimization: how far have we really come? AIAA J 52(4):670–690

    Article  Google Scholar 

  • Wang GG, Shan S (2007) Review of metamodeling techniques in support of engineering design optimization. J Mech Des 129(4):370–380

    Article  Google Scholar 

  • Xie F, Song W-P, Han Z-H (2008) Numerical study of high-resolution scheme based on preconditioning method. J Aircr 46(2):520–525

    Article  Google Scholar 

  • Zhang Y, Han Z-H., Liu J, Song W-P (2015) Efficient variable-fidelity optimization applied to benchmark transonic airfoil design. In: 7th Asia-Pac int Symp Aerosp Technol, Cairns, Australia, 25–27 November

  • Zhang Y, Han Z-H, Shi L-X, Song W-P (2016) Multi-round surrogate-based optimization for benchmark aerodynamic design problems. In: 54th AIAA Aerosp Sci Meet, AIAA paper 2016–1545, San Diego, US, 7–10 January

Download references

Acknowledgements

The author would like to thank Dr. Jun Liu and Prof. Wen-Ping Song for their valuable suggestion and discussion.

Funding

This research was sponsored by the National Natural Science Foundation of China (NSFC) under grant no. 11772261, Aeronautical Science Foundation of China under grant No. 2016ZA53011 and Innovation Foundation for Doctor Dissertation of Northwestern Polytechnical University under grant No. CX201801.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Zhong-Hua Han.

Appendix

Appendix

A.1 Hierarchical kriging

For an m dimensional problem, suppose we are concerned with the prediction of an expensive-to-evaluate (and unknown) hi-fi function y(x) : m → , with the assistance of a cheaper-to-evaluate low-fidelity function y(x) : m → . Assuming that the low- and hi-fi functions are observed at nlf and n sites, respectively, the sample datasets for a HK model are

$$ {\displaystyle \begin{array}{l}{\mathbf{S}}_{\mathrm{lf}}={\left[{\mathbf{x}}_{\mathrm{lf}}^{(1)},\dots, {\mathbf{x}}_{\mathrm{lf}}^{\left({n}_{\mathrm{lf}}\right)}\right]}^{\mathrm{T}}\in {\Re}^{n_{\mathrm{lf}}\times m},{\mathbf{y}}_{\mathrm{S},\mathrm{lf}}={\left[{y}_{\mathrm{lf}}^{(1)},\dots, {y}_{\mathrm{lf}}^{\left({n}_{\mathrm{lf}}\right)}\right]}^{\mathrm{T}}\in {\Re}^{n_{\mathrm{lf}}},\\ {}\mathbf{S}={\left[{\mathbf{x}}^{(1)},\dots, {\mathbf{x}}^{(n)}\right]}^{\mathrm{T}}\in {\Re}^{n\times m},{\mathbf{y}}_{\mathrm{S}}={\left[{y}^{(1)},\dots, {y}^{(n)}\right]}^{\mathrm{T}}\in {\Re}^n,\end{array}} $$
(22)

where the subscript “lf” denotes “low fidelity.”

Kriging model of cheap low-fi function

To build a surrogate model for the hi-fi and expensive function, we first build a surrogate model for the lower fidelity but cheaper function that will be used thereafter to assist the prediction. Assume a random process corresponding to the unknown low-fi function ylf(x)

$$ {Y}_{\mathrm{lf}}\left(\mathbf{x}\right)={\beta}_{0,\mathrm{lf}}+{Z}_{\mathrm{lf}}\left(\mathbf{x}\right), $$
(23)

where β0, lf is an unknown constant and Zlf(x) is a stationary random process. Then, we can follow (Sacks et al. 1989) to build a kriging based on the sampled data set  (Slf, yS, lf). After the kriging is fitted, the prediction of the low-fidelity function at any untried point x can be written as

$$ {\widehat{y}}_{\mathrm{lf}}\left(\mathbf{x}\right)={\beta}_{0,\mathrm{lf}}+{\mathbf{r}}_{\mathrm{lf}}^{\mathrm{T}}\left(\mathbf{x}\right){\mathbf{R}}_{\mathrm{lf}}^{-1}\left({\mathbf{y}}_{\mathrm{S},\mathrm{lf}}-{\beta}_{0,\mathrm{lf}}\mathbf{1}\right), $$
(24)

where \( {\beta}_{0,\mathrm{lf}}={\left({\mathbf{1}}^{\mathrm{T}}{\mathbf{R}}_{\mathrm{lf}}^{-1}\mathbf{1}\right)}^{-1}{\mathbf{1}}^{\mathrm{T}}{\mathbf{R}}_{\mathrm{lf}}^{-1}{\mathbf{y}}_{\mathrm{S},\mathrm{lf}} \); \( {\mathbf{R}}_{\mathrm{lf}}\in {\Re}^{n_{\mathrm{lf}}\times {n}_{\mathrm{lf}}} \) is the correlation matrix representing the correlation between the observed low-fi sample points; \( \mathbf{1}\in {\Re}^{n_{\mathrm{lf}}} \) is a column vector filled with ones; and \( {\mathbf{r}}_{\mathrm{lf}}\in {\Re}^{n_{\mathrm{lf}}} \) is the correlation vector representing the correlation between the untried point and the observed low-fi sample points. The MSE of the kriging prediction at any untried x, the uncertainty due to the lack of a low-fi sample, is

$$ \mathrm{MSE}\left[{\widehat{y}}_{\mathrm{lf}}\left(\mathbf{x}\right)\right]={s}_{\mathrm{lf}}^2\left(\mathbf{x}\right)={\sigma}_{\mathrm{lf}}^2\left[1.0-{\mathbf{r}}_{\mathrm{lf}}^{\mathrm{T}}{\mathbf{R}}_{\mathrm{lf}}^{-1}{\mathbf{r}}_{\mathrm{lf}}+{\left({\mathbf{r}}_{\mathrm{lf}}^{\mathrm{T}}{\mathbf{R}}_{\mathrm{lf}}^{-1}\mathbf{1}-1\right)}^2/{\mathbf{1}}^{\mathrm{T}}{\mathbf{R}}_{\mathrm{lf}}^{-1}\mathbf{1}\right]. $$
(25)

The reader is referred to (Simpson et al. 2001; Martin and Simpson 2005; Toal et al. 2008) for more details of building such a kriging.

Hierarchical kriging for expensive high-fidelity function

By directly taking the low-fi kriging multiplied by a scaling factor β0 as the model trend, the random process for the unknown hi-fi function is assumed as

$$ Y\left(\mathbf{x}\right)={\beta}_0{\widehat{y}}_{\mathrm{lf}}\left(\mathbf{x}\right)+Z\left(\mathbf{x}\right), $$
(26)

where β0 is a scaling factor representing how the low-fi kriging matches the hi-fi function; \( {\widehat{y}}_{\mathrm{lf}}\left(\mathbf{x}\right) \)denotes the prediction of low-fi kriging at x; and Z(x) is a stationary random process with zero mean. In this way, the trend of low-fi function is mapped to the sampled hi-fi data, resulting in a more accurate surrogate model for hi-fi function of interest. Then, a HK model (Han and Görtz 2012) can be built through the hi-fi sample datasets  (S, yS). By minimizing the MSE of the prediction, the HK prediction for the hi-fi and expensive function at any untried x can be written as

$$ \widehat{y}\left(\mathbf{x}\right)={\beta}_0{\widehat{y}}_{\mathrm{lf}}\left(\mathbf{x}\right)+{\mathbf{r}}^{\mathrm{T}}\left(\mathbf{x}\right){\mathbf{R}}^{-1}\left({\mathbf{y}}_{\mathrm{S}}-{\beta}_0\mathbf{F}\right), $$
(27)

where F ∈ n is a column vector filled with predictions of the low-fi kriging at the sites of hi-fi samples; R ∈ n × n is the correlation matrix representing the correlation between the observed hi-fi sample points; and r ∈ n is the correlation vector representing the correlation between the untried point and the observed hi-fi sample points

$$ {\displaystyle \begin{array}{l}{\beta}_0={\left({\mathbf{F}}^{\mathrm{T}}{\mathbf{R}}^{-1}\mathbf{F}\right)}^{-1}{\mathbf{F}}^{\mathrm{T}}{\mathbf{R}}^{-1}{\mathbf{y}}_{\mathrm{S}}\\ {}\mathbf{F}={\left[{\widehat{y}}_{\mathrm{lf}}\left({\mathbf{x}}^{(1)}\right),\cdots, {\widehat{y}}_{\mathrm{lf}}\left({\mathbf{x}}^{(n)}\right)\right]}^{\mathrm{T}}\in {\Re}^n,\\ {}\mathbf{R}=\left[\begin{array}{ccc}R\left({\mathbf{x}}^{(1)},{\mathbf{x}}^{(1)}\right)& \cdots & R\left({\mathbf{x}}^{(1)},{\mathbf{x}}^{(n)}\right)\\ {}\vdots & \ddots & \vdots \\ {}R\left({\mathbf{x}}^{(n)},{\mathbf{x}}^{(1)}\right)& \cdots & R\left({\mathbf{x}}^{(n)},{\mathbf{x}}^{(n)}\right)\end{array}\right]\in {\Re}^{n\times n},\\ {}\mathbf{r}={\left[R\left(\mathbf{x},{\mathbf{x}}^{(1)}\right),\dots, R\Big(\mathbf{x},{\mathbf{x}}^{(n)}\Big)\right]}^{\mathrm{T}}\in {\Re}^n,\end{array}} $$
(28)

where R(x, x) is the spatial correlation function which only depends on the Euclidean distance between the two sites x, x. Generally, the R(x, x) can be a Gaussian exponential function or a cubic spline function, see (Han and Görtz 2012).

The MSE of the HK prediction at any untried x, the uncertainty due to lack of a hi-fi sample, is

$$ \mathrm{MSE}\left\{\widehat{y}\left(\mathbf{x}\right)\right\}={s}^2\left(\mathbf{x}\right)={\sigma}^2\left\{1.0-{\mathbf{r}}^{\mathrm{T}}{\mathbf{R}}^{-1}\mathbf{r}+{\left[{\mathbf{r}}^{\mathrm{T}}{\mathbf{R}}^{-1}\mathbf{F}-{\widehat{y}}_{\mathrm{lf}}\left(\mathbf{x}\right)\right]}^2/{\left({\mathbf{F}}^{\mathrm{T}}{\mathbf{R}}^{-1}\mathbf{F}\right)}^{-1}\right\}. $$
(29)

In comparison to a cokriging model, the HK model does not need to calculate the cross covariance between low- and hi-fi samples. As a result, the correlation matrix of a HK model is relatively smaller. In addition, the HK model can provide a more reasonable MSE estimation than any of the existing kriging and cokriging models (Han and Görtz 2012), which is very beneficial for infill-sampling based on the method such as EI.

A.2 Standard expected improvement method

For using the expected improvement (EI) method proposed by Jones et al. (1998), we can assume that the prediction of the HK model at any untried site x obeys a normal distribution \( \widehat{Y}\left(\mathbf{x}\right)\sim N\left[\widehat{y}\left(\mathbf{x}\right),{s}^2\left(\mathbf{x}\right)\right] \), with the mean being the surrogate prediction \( \widehat{y}\left(\mathbf{x}\right) \) and the standard deviation s(x) being its root mean-squared error (RMSE). Then the statistical improvement at any untried location w.r.t. the best hi-fi objective function observed so far ymin is defined as:

$$ I\left(\mathbf{x}\right)=\max \left({y}_{\mathrm{min}}-\widehat{Y}\left(\mathbf{x}\right),0\right). $$
(30)

Then, the EI function can be written as

$$ EI\left(\mathbf{x}\right)=\left\{\begin{array}{cc}\left({y}_{\mathrm{min}}-\widehat{y}\left(\mathbf{x}\right)\right)\Phi \left(\frac{y_{\mathrm{min}}-\widehat{y}\left(\mathbf{x}\right)}{s\left(\mathbf{x}\right)}\right)+s\left(\mathbf{x}\right)\phi \left(\frac{y_{\mathrm{min}}-\widehat{y}\left(\mathbf{x}\right)}{s\left(\mathbf{x}\right)}\right),& if\kern0.8000001em s\left(\mathbf{x}\right)>0\\ {}0,& if\kern0.8000001em s\left(\mathbf{x}\right)=0\end{array}\right., $$
(31)

where Φ and ϕ are the cumulative distribution function and probability density function of standard normal distribution, respectively.

For a constrained optimization, the HK model for the constraint function, \( \widehat{g}\left(\mathbf{x}\right) \), is also built. We can also assume the prediction at any untried site obeys a normal distributed, \( \widehat{G}\left(\mathbf{x}\right)\sim N\left[\widehat{g}\left(\mathbf{x}\right),{s}_g^2\left(\mathbf{x}\right)\right] \), with the mean value being the predictor \( \widehat{g}\left(\mathbf{x}\right) \) and the standard deviation being its RMSE sg(x). Therefore, the probability of satisfying the constraint is

$$ P\left[\widehat{G}\left(\mathbf{x}\right)\le 0\right]=\Phi \left(\frac{-\widehat{g}\left(\mathbf{x}\right)}{s_g\left(\mathbf{x}\right)}\right). $$
(32)

And the constrained EI function can be given by

$$ CEI\left(\mathbf{x}\right)=E\left[I\left(\mathbf{x}\right)\cap \left[G\left(\mathbf{x}\right)\le 0\right]\right]= EI\left(\mathbf{x}\right)\cdot P\left[G\left(\mathbf{x}\right)\le 0\right]. $$
(33)

If there are NC constraints, we should built NC HK models for every constraint function, and the resulting constrained EI function is

$$ CEI\left(\mathbf{x}\right)= EI\left(\mathbf{x}\right)\cdot {\prod}_{i=1}^{N_{\mathrm{C}}}P\left[{G}_i\left(\mathbf{x}\right)\le 0\right]. $$
(34)

Then the following unconstrained sub-optimization problem is formulated as

$$ \mathbf{x}=\underset{{\mathbf{x}}_{low}\le \mathbf{x}\le {\mathbf{x}}_{up}}{\arg \max } CEI\left(\mathbf{x}\right). $$
(35)

A hybrid method of combing GA, Hooke and Jeeves pattern search, and BFGS gradient-based method is used to solve the above sub-optimization problem to suggest the new sample point (Han 2016a, b), which is to be evaluate by hi-fi numerical analysis again. Please note only the hi-fi sample point is obtained here, since the above EI function is defined based on the uncertainty due to lack of a hi-fi sample. Therefore, if we need to adaptively select low-fi sample points as well, the EI function based on the uncertainty coming from the low-fi kriging model should be formulated, which is almost a blank area so far and will be done in Section 3.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Zhang, Y., Han, ZH. & Zhang, KS. Variable-fidelity expected improvement method for efficient global optimization of expensive functions. Struct Multidisc Optim 58, 1431–1451 (2018). https://doi.org/10.1007/s00158-018-1971-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00158-018-1971-x

Keywords

  • Efficient global optimization
  • Surrogate model
  • Variable-fidelity optimization
  • Expected improvement
  • Kriging