Skip to main content
SpringerLink
Log in

A sparse multi-fidelity surrogate-based optimization method with computational awareness

  • Original Article
  • Published:
Engineering with Computers Aims and scope Submit manuscript

Abstract

CoKriging is a popular surrogate modeling approach to approximate the input–output relationship using multi-fidelity data from different sources. However, it suffers from the big data issue due to its cubic time complexity and square memory complexity with the data size. This becomes even exacerbated for iterative design optimization. To overcome this limitation, this paper presents a new data sparsification method for multi-fidelity surrogate-based optimization (MFSBO). It includes two key components: reduced design space and data filtering (RDS&DF), which alleviate the surrogate modeling complexity and time for improved efficiency while balancing between exploration and exploitation during optimization. RDS&DF is also combined with an expected improvement reduction (EIR)-based infill technique (Yang et al. in Struct Multidiscip Optim, 2022. https://doi.org/10.1007/s00158-022-03240-x), enabling both parsimony and computational awareness for MFSBO. Two case studies are conducted to examine the proposed method. Results demonstrate that a significant reduction in the modeling and thus optimization time (69.49% and 73.92%) is achieved while retaining the design accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Forrester AIJ, Sóbester A, Keane AJ (2007) Multi-fidelity optimization via surrogate modelling. Proc R Soc A Math Phys Eng Sci 463:3251–3269. https://doi.org/10.1098/rspa.2007.1900

    Article  MathSciNet  MATH  Google Scholar 

  2. Yang H, Hong SH, ZhG R, Wang Y (2020) Surrogate-based optimization with adaptive sampling for microfluidic concentration gradient generator design. RSC Adv 10:13799–13814

    Article  Google Scholar 

  3. Park C, Haftka RT, Kim NH (2017) Remarks on multi-fidelity surrogates. Struct Multidiscip Optim 55:1029–1050. https://doi.org/10.1007/s00158-016-1550-y

    Article  MathSciNet  Google Scholar 

  4. Kleijnen JPC (2017) Design and analysis of simulation experiments: tutorial. Adv Model Simul 2017:135–158

  5. Marques AN, Lam RR, Chaudhuri A et al (2019) A multifidelity method for locating aeroelastic flutter boundaries. AIAA Scitech 2019 Forum. https://doi.org/10.2514/6.2019-0438

  6. Fernández-Godino MG, Park C, Kim N-H, Haftka RT (2016) Review of multi-fidelity models. Preprint arXiv:16090.7196

  7. Peherstorfer B, Willcox K, Gunzburger M (2018) Survey of multifidelity methods in uncertainty propagation, inference, and optimization. SIAM Rev 60:550–591. https://doi.org/10.1137/16M1082469

    Article  MathSciNet  MATH  Google Scholar 

  8. Liu H, Ong Y, Shen X, Cai J (2020) When Gaussian process meets big data: a review of scalable GPs. IEEE Trans Neural Netw Learn Syst. https://doi.org/10.1109/tnnls.2019.2957109

    Article  MathSciNet  Google Scholar 

  9. Lawrence N, Seeger M, Herbrich R (2003) Fast sparse Gaussian process methods: the informative vector machine. Adv Neural Inf Process Syst 1:1

    Google Scholar 

  10. Lee BJ, Lee J, Kim KE (2017) Hierarchically-partitioned Gaussian process approximation. In: Proceedings of the 20th international conference on artificial intelligence and statistics, AISTATS 2017 54

  11. Hensman J, Fusi N, Lawrence ND (2013) Gaussian processes for big data. Preprint arXiv:1309.6835

  12. Snelson E, Ghahramani Z (2007) Local and global sparse Gaussian process approximations. J Mach Learn Res 2:524–531

    Google Scholar 

  13. Kleijnen JPC, van Beers WCM (2020) Prediction for big data through Kriging: small sequential and one-shot designs. Am J Math Manag Sci 39:199–213. https://doi.org/10.1080/01966324.2020.1716281

    Article  Google Scholar 

  14. Chalupka K, Williams CKI, Murray I (2013) A framework for evaluating approximation methods for Gaussian process regression. J Mach Learn Res 14:333–350

    MathSciNet  MATH  Google Scholar 

  15. Das S, Roy S, Sambasivan R (2018) Fast Gaussian process regression for big data. Big Data Res 14:12–26. https://doi.org/10.1016/j.bdr.2018.06.002

    Article  Google Scholar 

  16. Melkumyan A, Ramos FT (2009) A sparse covariance function for exact Gaussian process inference in large datasets. In: 21st international joint conference on artificial intelligence

  17. Wilson A, Nickisch H (2015) Kernel interpolation for scalable structured Gaussian processes (KISS-GP). In: International conference on machine learning. PMLR, pp 1775–1784

  18. Rullière D, Durrande N, Bachoc F, Chevalier C (2018) Nested Kriging predictions for datasets with a large number of observations. Stat Comput 28:849–867

    Article  MathSciNet  MATH  Google Scholar 

  19. Gramacy RB (2016) LaGP: large-scale spatial modeling via local approximate Gaussian processes in R. J Stat Softw. https://doi.org/10.18637/jss.v072.i01

  20. Wang H, Emmerich M, Van Stein B, Back T (2017) Time complexity reduction in efficient global optimization using cluster Kriging. In: GECCO 2017—proceedings of the 2017 genetic and evolutionary computation conference 889–896. https://doi.org/10.1145/3071178.3071321

  21. Yang H, Hong SH, Wang Y (2022) A sequential multi-fidelity surrogate-based optimization methodology based on expected improvement reduction. Struct Multidiscip Optim. https://doi.org/10.1007/s00158-022-03240-x

    Article  Google Scholar 

  22. Yang H, Hong SH, Wang Y (2022) A novel adaptive sampling method based on expected improvement reduction. In: Conference proceedings—IEEE SOUTHEASTCON

  23. Shi R, Liu L, Long T et al (2018) Dual-sampling based co-kriging method for design optimization problems with multi-fidelity models. Multidiscip Anal Optim Conf 2018:1–14. https://doi.org/10.2514/6.2018-3747

    Article  Google Scholar 

  24. Forrester AIJ, Sóbester A, Keane AJ (2008) Engineering design via surrogate modelling. Wiley, New York

  25. Laguna M, Marti R (2005) Experimental testing of advanced scatter search designs for global optimization of multimodal functions. J Global Optim 33:235–255

    Article  MathSciNet  MATH  Google Scholar 

  26. Irimia D, Geba DA, Toner M (2006) Universal microfluidic gradient generator. Anal Chem 78:3472–3477. https://doi.org/10.1021/ac0518710

    Article  Google Scholar 

  27. Cabaleiro JM (2020) Flowrate independent 3D printed microfluidic concentration gradient generator. Chem Eng J 382:122742

    Article  Google Scholar 

  28. Yang CG, Wu YF, Xu ZR, Wang JH (2011) A radial microfluidic concentration gradient generator with high-density channels for cell apoptosis assay. Lab Chip 11:3305–3312. https://doi.org/10.1039/c1lc20123a

    Article  Google Scholar 

  29. Wang X, Liu Z, Pang Y (2017) Concentration gradient generation methods based on microfluidic systems. RSC Adv 7:29966–29984. https://doi.org/10.1039/c7ra04494a

    Article  Google Scholar 

  30. Hong SH, Yang H, Wang Y (2020) Inverse design of microfluidic concentration gradient generator using deep learning and physics-based component model. Microfluid Nanofluidics 24:1

  31. Wang Y, Mukherjee T, Lin Q (2006) Systematic modeling of microfluidic concentration gradient generators. J Micromech Microeng 16:2128–2137. https://doi.org/10.1088/0960-1317/16/10/029

    Article  Google Scholar 

  32. Zhou Y, Wang Y, Mukherjee T, Lin Q (2009) Generation of complex concentration profiles by partial diffusive mixing in multi-stream laminar flow. Lab Chip 9:1439–1448. https://doi.org/10.1039/b818485b

    Article  Google Scholar 

Download references

Acknowledgements

YW acknowledges the faculty startup grant from the University of South Carolina for partial funding of this research.

Author information

Authors and Affiliations

  1. Department of Mechanical Engineering, University of South Carolina, Columbia, SC, 29208, USA

    Haizhou Yang & Yi Wang

Authors
  1. Haizhou Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yi Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yi Wang.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Data availability

All data and codes are available from the author upon reasonable request.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, H., Wang, Y. A sparse multi-fidelity surrogate-based optimization method with computational awareness. Engineering with Computers 39, 3473–3489 (2023). https://doi.org/10.1007/s00366-022-01766-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00366-022-01766-8

Keywords

Search

Navigation