Skip to main content
SpringerLink
Log in

A multi-fidelity Bayesian optimization approach based on the expected further improvement

  • Research Paper
  • Published:
Structural and Multidisciplinary Optimization Aims and scope Submit manuscript

Abstract

Sampling efficiency is important for simulation-based design optimization. While Bayesian optimization (BO) has been successfully applied in engineering problems, the cost associated with large-scale simulations has not been fully addressed. Extending the standard BO approaches to multi-fidelity optimization can utilize the information of low-fidelity models to further reduce the optimization cost. In this work, a multi-fidelity Bayesian optimization approach is proposed, in which hierarchical Kriging is used for constructing the multi-fidelity metamodel. The proposed approach quantifies the effect of HF and LF samples in multi-fidelity optimization based on a new concept of expected further improvement. A novel acquisition function is proposed to determine both the location and fidelity level of the next sample simultaneously, with the consideration of balance between the value of information provided by the new sample and the associated sampling cost. The proposed approach is compared with some state-of-the-art methods for multi-fidelity global optimization with numerical examples and an engineering case. The results show that the proposed approach can obtain global optimal solutions with reduced computational costs.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  • Bakr MH, Bandler JW, Madsen K, Søndergaard J (2001) An introduction to the space mapping technique. Optim Eng 2:369–384

    Article  MathSciNet  Google Scholar 

  • Bandler JW, Biernacki RM, Chen SH, Grobelny PA, Hemmers RH (1994) Space mapping technique for electromagnetic optimization. IEEE Trans Microw Theory Tech 42:2536–2544

    Article  Google Scholar 

  • Cai X, Qiu H, Gao L, Yang P, Shao X (2016) An enhanced RBF-HDMR integrated with an adaptive sampling method for approximating high dimensional problems in engineering design. Struct Multidiscip Optim 53:1209–1229

    Article  Google Scholar 

  • Chang KJ, Haftka RT, Giles GL, Kao I-J (1993) Sensitivity-based scaling for approximating structural response. J Aircr 30:283–288

    Article  Google Scholar 

  • Coello CAC (2000) Use of a self-adaptive penalty approach for engineering optimization problems. Comput Ind 41:113–127

    Article  Google Scholar 

  • Courrier N, Boucard P-A, Soulier B (2016) Variable-fidelity modeling of structural analysis of assemblies. J Glob Optim 64:577–613

    Article  MathSciNet  Google Scholar 

  • Gano SE, Renaud JE, Sanders B (2005) Hybrid variable fidelity optimization by using a kriging-based scaling function. AIAA J 43:2422–2433

    Article  Google Scholar 

  • Gardner JR, Kusner MJ, Xu ZE, Weinberger KQ, Cunningham JP (2014) Bayesian optimization with inequality constraints. In: ICML. pp. 937–945

  • Gelbart MA, Snoek J, Adams RP (2014) Bayesian optimization with unknown constraints. arXiv preprint arXiv:1403.5607

  • Ghoreishi SF, Allaire D (2019) Multi-information source constrained Bayesian optimization. Struct Multidiscip Optim 59:977–991

    Article  MathSciNet  Google Scholar 

  • Ghoreishi SF, Molkeri A, Srivastava A, Arroyave R, Allaire D (2018) Multi-information source fusion and optimization to realize ICME: application to dual-phase materials. J Mech Des 140:111409

    Article  Google Scholar 

  • Ghoreishi SF, Molkeri A, Arróyave R, Allaire D, Srivastava A (2019) Efficient use of multiple information sources in material design. Acta Mater 180:260–271

    Article  Google Scholar 

  • Han Z-H, Görtz S (2012) Hierarchical kriging model for variable-fidelity surrogate modeling. AIAA J 50:1885–1896

    Article  Google Scholar 

  • Huang D, Allen TT, Notz WI, Miller RA (2006) Sequential kriging optimization using multiple-fidelity evaluations. Struct Multidiscip Optim 32:369–382

    Article  Google Scholar 

  • Jones DR (2001) A taxonomy of global optimization methods based on response surfaces. J Glob Optim 21:345–383

    Article  MathSciNet  Google Scholar 

  • Jones DR, Schonlau M, Welch WJ (1998) Efficient global optimization of expensive black-box functions. J Glob Optim 13:455–492

    Article  MathSciNet  Google Scholar 

  • Kennedy MC, O'Hagan A (2000) Predicting the output from a complex computer code when fast approximations are available. Biometrika 87:1–13

    Article  MathSciNet  Google Scholar 

  • Kim Y, Lee S, Yee K, Rhee D-H (2017) High-to-low initial sample ratio of hierarchical kriging for film hole array optimization. J Propuls Power 34:108–115

    Article  Google Scholar 

  • Koziel S, Bandler JW, Madsen K (2006) A space-mapping framework for engineering optimization—theory and implementation IEEE transactions on microwave. Theory Tech 54:3721–3730

    Article  Google Scholar 

  • Liu Y, Chen S, Wang F, Xiong F (2018) Sequential optimization using multi-level co-Kriging and extended expected improvement criterion. Struct Multidiscip Optim 58:1155–1173

    Article  MathSciNet  Google Scholar 

  • McIntire M, Ratner D, Ermon S. (2016). Sparse Gaussian processes for Bayesian optimization. Proceedings of the Thirty-Second Conference on Uncertainty in Artificial Intelligence, pp. 517-526

  • Palar PS, Shimoyama K (2017) Multi-fidelity uncertainty analysis in CFD using hierarchical kriging. In: 35th AIAA Applied Aerodynamics Conference, p. 3261

  • Park J-S (1994) Optimal Latin-hypercube designs for computer experiments. J Stat Plan Inference 39:95–111

    Article  MathSciNet  Google Scholar 

  • Queipo NV, Haftka RT, Shyy W, Goel T, Vaidyanathan R, Tucker PK (2005) Surrogate-based analysis and optimization. Prog Aerosp Sci 41:1–28

    Article  Google Scholar 

  • Schonlau M, Welch WJ, Jones DR (1998) Global versus local search in constrained optimization of computer models. Lecture Notes-Monograph Series 11-25

  • Shu L, Jiang P, Wan L, Zhou Q, Shao X, Zhang Y (2017) Metamodel-based design optimization employing a novel sequential sampling strategy. Eng Comput 34:2547–2564

    Article  Google Scholar 

  • Shu L, Jiang P, Song X, Zhou Q (2019a) Novel approach for selecting low-fidelity scale factor in multifidelity metamodeling. AIAA J 57:5320–5330

    Article  Google Scholar 

  • Shu L, Jiang P, Zhou Q, Xie T (2019b) An online variable-fidelity optimization approach for multi-objective design optimization. Struct Multidiscip Optim 60:1059–1077

    Article  MathSciNet  Google Scholar 

  • Song C, Song W, Yang X (2017) Gradient-enhanced hierarchical kriging model for aerodynamic design optimization. J Aerosp Eng 30:04017072

    Article  Google Scholar 

  • Tran A, Sun J, Furlan JM, Pagalthivarthi KV, Visintainer RJ, Wang Y (2019a) pBO-2GP-3B: a batch parallel known/unknown constrained Bayesian optimization with feasibility classification and its applications in computational fluid dynamics. Comput Methods Appl Mech Eng 347:827–852

    Article  MathSciNet  Google Scholar 

  • Tran A, Tran M, Wang Y (2019b) Constrained mixed-integer Gaussian mixture Bayesian optimization and its applications in designing fractal and auxetic metamaterials. Struct Multidiscip Optim 59:2131–2154

    Article  MathSciNet  Google Scholar 

  • Tran A, Wildey T, McCann S (2020) sMF-BO-2CoGP: a sequential multi-fidelity constrained Bayesian optimization framework for design applications. J Comput Inf Sci Eng 20:031007

    Article  Google Scholar 

  • Ulaganathan S, Couckuyt I, Ferranti F, Laermans E, Dhaene T (2015) Performance study of multi-fidelity gradient enhanced kriging. Struct Multidiscip Optim 51:1017–1033

    Article  Google Scholar 

  • Wang GG (2003) Adaptive response surface method using inherited Latin hypercube design points. J Mech Des 125:210–220

    Article  Google Scholar 

  • Wang GG, Shan S (2007) Review of metamodeling techniques in support of engineering design optimization. J Mech Des 129:370–380

    Article  Google Scholar 

  • Xiong Y, Chen W, Tsui K-L (2008) A new variable-fidelity optimization framework based on model fusion and objective-oriented sequential sampling. J Mech Des 130:111401

    Article  Google Scholar 

  • Zhang Y, Han Z-H, Liu J, Song W-P (2015) Efficient variable-fidelity optimization applied to benchmark transonic airfoil design. In: 7th Asia-Pac int Symp Aerosp Technol, Cairns, Australia, pp. 25–27

  • Zhang Y, Han Z-H, Zhang K-S (2018) Variable-fidelity expected improvement method for efficient global optimization of expensive functions. Struct Multidiscip Optim 58:1431–1451

    Article  MathSciNet  Google Scholar 

  • Zhang J, Yao X, Liu M, Wang Y. (2019) A Bayesian discrete optimization algorithm for permutation problems. Proceedings of 2019 IEEE Symposium Series on Computational Intelligence (SSCI 2019), pp.871-881

  • Zhou Q, Shao X, Jiang P, Zhou H, Shu L (2015) An adaptive global variable fidelity metamodeling strategy using a support vector regression based scaling function. Simul Model Pract Theory 59:18–35

    Article  Google Scholar 

  • Zhou Q, Shao X, Jiang P, Gao Z, Wang C, Shu L (2016) An active learning metamodeling approach by sequentially exploiting difference information from variable-fidelity models. Adv Eng Inform 30:283–297

    Article  Google Scholar 

  • Zhou Q, Wang Y, Choi S-K, Jiang P, Shao X, Hu J (2017) A sequential multi-fidelity metamodeling approach for data regression. Knowl-Based Syst 134:199–212

    Article  Google Scholar 

Download references

Acknowledgements

This research was supported by the National Natural Science Foundation of China (Grant No. 51775203, 51805179, and 51721092). The support of the China Scholarship Council is also appreciated.

Author information

Authors and Affiliations

  1. The State Key Laboratory of Digital Manufacturing Equipment and Technology, School of Mechanical Science and Engineering, Huazhong University of Science and Technology, Wuhan, 430074, People’s Republic of China

    Leshi Shu & Ping Jiang

  2. Woodruff School of Mechanical Engineering, Georgia Institute of Technology, 813 Ferst Drive NW, Atlanta, GA, 30332-0405, USA

    Leshi Shu & Yan Wang

Authors
  1. Leshi Shu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ping Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yan Wang.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Replication of results

Data or implementation code are available upon request.

Additional information

Responsible Editor: Nestor V Queipo

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shu, L., Jiang, P. & Wang, Y. A multi-fidelity Bayesian optimization approach based on the expected further improvement. Struct Multidisc Optim 63, 1709–1719 (2021). https://doi.org/10.1007/s00158-020-02772-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00158-020-02772-4

Keywords

Search

Navigation