Abstract
To reduce the computational cost, multi-fidelity (MF) metamodel methods have been widely used in engineering optimization. Most of these methods are based on the standard Gaussian random process theory; thus, the time cost required for hyperparameter estimation increases significantly with an increase in the dimension and nonlinearity of the problems especially for high-dimensional problems. To address these issues, by exploiting the great potential of deep neural networks in high-dimensional information extraction and approximation, a meta-learning-based multi-fidelity Bayesian neural network (ML-MFBNN) method is developed in this study. Based on this, to further reduce the computational cost, an adaptive multi-fidelity sampling strategy is proposed in combination with Bayesian deep learning to sequentially select the highly cost-effective samples. The effectiveness and advantages of the proposed MF-MFBNN and adaptive multi-fidelity sampling strategy are verified through eight mathematical examples, and the application to model validation of computational fluid dynamics and robust shape optimization of the ONERA M6 wing.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Abbreviations
- \(\nabla\) :
-
Gradient
- \(\mathcal{L}\) :
-
Loss function
- \(\theta\) :
-
Network parameters
- \(\varphi\) :
-
Network initial parameters
- \(\theta^{ - }\) :
-
Parameters of the network other than the output layer
- \({\mathbf{w}}^{L}\) :
-
The weights of the Lth layer of the network
- \({\mathbf{b}}^{L}\) :
-
The deviations of the Lth layer of the network
- x:
-
Input vector
- y:
-
Response vector
- \(D\) :
-
Sample dataset
- \(f\left( \cdot \right)\) :
-
Bayesian neural networks
- \(f_{{{\varvec{\uptheta}}}} \left( {\mathbf{x}} \right)\) :
-
Output of Bayesian neural networks
- HF:
-
High-Fidelity
- LF:
-
Low-Fidelity
- ML:
-
Meta-learning
- BNN:
-
Bayesian Neural Network
- MF:
-
Multi-fidelity
References
Aute V, Saleh K, Abdelaziz O, Azarm S, Radermacher R (2013) Cross-validation based single response adaptive design of experiments for Kriging metamodeling of deterministic computer simulations. Struct Multidisc Optim 48(3):581–605. https://doi.org/10.1007/s00158-013-0918-5
Bailly J, Bailly D (2019) Multifidelity aerodynamic optimization of a helicopter rotor blade. AIAA J 57(8):3132–3144. https://doi.org/10.2514/1.J056513
Blundell C, Cornebise J, Kavukcuoglu K, Wierstra D (2015) Weight uncertainty in neural network. Int Conf Mach Learning 37:1613–1622. https://doi.org/10.5555/3045118.3045290
Bouhlel MA, He S, Martins JRRA (2020) Scalable gradient–enhanced artificial neural networks for airfoil shape design in the subsonic and transonic regimes. Struct Multidisc Optim 61(4):1363–1376. https://doi.org/10.1007/s00158-020-02488-5
Chakraborty S (2021) Transfer learning based multi-fidelity physics informed deep neural network”. J Comput Phys 426:109942. https://doi.org/10.1016/j.jcp.2020.109942
Chen S, Jiang Z, Yang S, Chen W (2017) Multi-model fusion based sequential optimization. AIAA J 55(1):241–254. https://doi.org/10.2514/1.J054729
Deng Z, Bi S, Atamturktur S (2014) Stochastic model updating using distance discrimination analysis. Chin J Aeronaut 27(5):1188–1198. https://doi.org/10.1016/j.cja.2014.08.008
Ellis AG, Iskandar R, Schmid CH, Wong JB, Trikalinos TA (2020) Active learning for efficiently training emulators of computationally expensive mathematical models. Stats Med 39(25):3521–3548. https://doi.org/10.1002/sim.8679
Eweis-Labolle JT, Oune N, Bostanabad R (2022) Data fusion with latent map Gaussian processes. J Mech Des 144(9):091703. https://doi.org/10.1115/1.4054520
Finn, C., Abbeel, P., and Levine, S. 2017. “Model-agnostic meta-learning for fast adaptation of deep networks,” Proceedings of the 34th International Conference on Machine Learning. 70: 1126–1135. https://doi.org/10.5555/3305381.3305498
Forrester, A. I. J., Sóbester, A., and Keane, A. J. 2007. Multi-fidelity optimization via surrogate modelling. Proceedings of the royal society a: mathematical, physical and engineering sciences. 463(2088): 3251-3269. https://doi.org/10.1098/rspa.2007.1900
Halder R, Damodaran M, Khoo BC (2020) Deep learning based reduced order model for airfoil-gust and aeroelastic interaction. AIAA J 58(10):4304–4321. https://doi.org/10.2514/1.J059027
Han Z, Görtz S (2012) Hierarchical kriging model for variable-fidelity surrogate modeling. AIAA J 50(9):1885–1896. https://doi.org/10.2514/1.J051354
He X, Zhao F, Vahdati M (2020) Uncertainty quantification of Spalart-Allmaras turbulence model coefficients for simplified compressor flow features. J Fluids Eng 142(9):091501. https://doi.org/10.1115/1.4047026
Huang D, Allen TT, Notz WI, Miller RA (2006) Sequential kriging optimization using multiple-fidelity evaluations. Struct Multidiscip Optim 32(5):369–382. https://doi.org/10.1007/s00158-005-0587-0
Jivani A, Huan X, Safta C, Zhou BY, Gauger NR (2021) Uncertainty quantification for a turbulent round jet using multifidelity karhunen-loeve expansions. AIAA Scitech. https://doi.org/10.2514/6.2021-1367
Jones DR, Schonlau M, Welch WJ (1998) Efficient global optimization of expensive black-box functions. J Global Optim 13(4):455–492. https://doi.org/10.1023/A:1008306431147
Lee SH, Chen W (2009) A comparative study of uncertainty propagation methods for black-box type problems. Struct Multidiscip Optim 37(3):239–253. https://doi.org/10.1007/s00158-008-0234-7
Li J, Zhang M, Martins JR, Shu C (2020) Efficient aerodynamic shape optimization with deep-learning-based geometric filtering. AIAA J 58(10):4243–4259. https://doi.org/10.2514/1.J059254
Liu Y, Chen S, Wang F, Xiong F (2018) Sequential optimization using multi-level cokriging and extended expected improvement criterion. Struct Multidiscip Optim 58(3):1155–1173. https://doi.org/10.1007/s00158-018-1959-6
Liu H, Ong YS, Shen X, Cai J (2020) When Gaussian process meets big data: a review of scalable GPs. IEEE Trans Neural Networks Learning Syst 31(11):4405–4423. https://doi.org/10.1109/TNNLS.2019.2957109
Lv L, Shi M, Song X, Sun W, Zhang J (2020) A fast-converging ensemble infilling approach balancing global exploration and local exploitation: the go-inspired hybrid infilling strategy. J Mech Des 142(2):021403. https://doi.org/10.1115/1.4044112
Meng XH, Karniadakis GEA (2020) A composite neural network that learns from multi-fidelity data: application to function approximation and inverse PDE problems. J Comput Phys 401:109020. https://doi.org/10.1016/j.jcp.2019.109020
Meng X, Babaee H, Karniadakis GE (2021) Multi-fidelity Bayesian neural networks: algorithms and applications. J Comput Phys 438:110361. https://doi.org/10.1016/j.jcp.2021.110361
Paiva RM, Crawford C, Suleman A (2014) Robust and reliability-based design optimization framework for wing design. AIAA J 52(4):711–724. https://doi.org/10.2514/1.J052161
Panaretos VM, Zemel Y (2019) Statistical aspects of wasserstein distances. Ann Rev Stat App 6:405–431. https://doi.org/10.1146/annurev-statistics-030718-104938
Park GJ, Lee TH, Lee KH, Hwang KH (2006) Robust design: an overview. AIAA J 44(1):181–191. https://doi.org/10.2514/1.13639
Parussini L, Venturi D, Perdikaris P, Karniadakis GE (2017) Multi-fidelity Gaussian process regression for prediction of random fields. J Comput Phys 336:36–50. https://doi.org/10.1016/j.jcp.2017.01.047
Perdikari, P., Raissi, M., Damianou, A., Lawrence, N. D., and Karniadakis, G. E. 2017. Nonlinear information fusion algorithms for data-efficient multi-fidelity modelling,” Proceedings of the Royal Society A: Mathematical, Physical and Engineering Science. http://dx.doi.org/https://doi.org/10.1098/rspa.2016.0751
Ren C, Xiong F, Wang F, Mo B, Hu Z (2021) A maximum cost-performance sampling strategy for multi-fidelity PC-Kriging. Struct Multidiscip Optim 64(6):3381–3399. https://doi.org/10.1007/s00158-021-02994-0
Rodríguez P, Laradji I, Drouin A, Lacoste A (2020) “Embedding propagation: smoother manifold for few-shot classification. European Conf Comput vis. https://doi.org/10.1007/978-3-030-58574-7_8
Schmitt, V., and F. Charpin, “Pressure distributions on the ONERA-M6-Wing at transonic mach numbers,” Experimental Data Base for Computer Program Assessment. AGARD AR 138, May 1979. https://www.grc.nasa.gov/www/wind/valid/m6wing/m6wing.html
Shah MZM, Basuno B, Abdullah A (2020) "Comparative study on several type of turbulence model available in ansy-fluent software for ONERA m6 wing aerodynamic analysis. J Adv Mech Eng App 1(1):9–19. https://doi.org/10.30880/jamea.2020.01.01.002
Snoek J, Rippel O, Swersky K, Kiros R, Satish N (2015) Scalable bayesian optimization using deep neural networks. Int Conf Mach Learning, PMLR. https://doi.org/10.48550/arXiv.1502.05700
Tao J, Sun G (2019) Application of deep learning based multi-fidelity surrogate model to robust aerodynamic design optimization. Aerosp Sci Technol 92:722–737. https://doi.org/10.1016/j.ast.2019.07.002
Tripathy RK, Bilionis I (2018) Deep UQ: learning deep neural network surrogate models for high dimensional uncertainty quantification. J Comput Phys 375:565–588. https://doi.org/10.1016/j.jcp.2018.08.036
Wang H, Yeung DY (2021) A survey on Bayesian deep learning. ACM Comput Surv (CSUR) 53(5):1–37. https://doi.org/10.1145/3409383
Wang F, Xiong F, Chen S, Song J (2019) Multi-fidelity uncertainty propagation using polynomial chaos and Gaussian process modeling. Struct Multidisci Optim 60(4):1583–1604. https://doi.org/10.1007/s00158-019-02287-7
Xu J, Du Q (2020) Learning transferable features in meta-learning for few-shot text classification. Pattern Recogn Lett 135:271–278. https://doi.org/10.1016/j.patrec.2020.05.007
Yin T, Zhu HP (2020) An efficient algorithm for architecture design of Bayesian neural network in structural model updating. Comput-Aided Civil Infra Eng 35(4):354–372. https://doi.org/10.1111/mice.12492
Zanjani Foumani S, Shishehbor M, Yousefpour A, Bostanabad R., “Multi-Fidelity Cost-Aware Bayesian Optimization”, 2022. https://doi.org/10.48550/arXiv.2211.02732
Zhang Y, Tao S, Chen W, Daniel W (2020) A latent variable approach to Gaussian process modeling with qualitative and quantitative factors. Technometrics 62(3):291–302. https://doi.org/10.1080/00401706.2019.1638834
Zhang X, Xie F, Ji T, Zhu Z, Zheng Y (2021a) Multi-fidelity deep neural network surrogate model for aerodynamic shape optimization. Comput Method Appl Mech Eng 373:113485. https://doi.org/10.1016/j.cma.2020.113485
Zhang X, Xie F, Ji T, Zhu Z, Zheng Y (2021b) Multi-fidelity deep neural network surrogate model for aerodynamic shape optimization. Comput Methods App Mech Eng 373:113485. https://doi.org/10.1016/j.cma.2020.113485
Zhang S, Liang P, Pang Y, Li J, Song X (2022) Multi-fidelity surrogate model ensemble based on feasible intervals. Struct Multidisci Optim 60(4):1583–2160. https://doi.org/10.1007/s00158-022-03329-3
Zhou Q, Wang Y, Choi SK, Jiang P, Shao X, Hu J (2017) A sequential multi-fidelity metamodeling approach for data regression. Knowl-Based Syst 134(15):199–212. https://doi.org/10.1016/j.knosys.2017.07.033
Zhou Q, Yang Y, Song X, Han Z, Cheng Y, Hu J, Shu L (2020) Survey of Multi-fidelity surrogate models and their applications in the design and optimization of engineering equipment. J Mech Eng 56(24):219–245. https://doi.org/10.3901/JME.2020.24.219
Acknowledgements
The work was supported by National Natural Science Foundation of China [grant number 52175214] and Basic Research Program of Equipment Development Department [grant number 514010103-302].
Funding
National Natural Science Foundation of China, 52175214; The basic research program, 514010103-302
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Replication of results
The results shown in the manuscript can be reproduced. Example in Sect. 4.1 was uploaded to as supplementary material. The remaining examples are easy to implement by changing the response functions and samples based on the codes provided to obtain the results shown in the manuscript.
Ethical approval
This article does not contain any studies with human participants or animals performed by any of the authors.
Additional information
Responsible Editor: Ramin Bostanabad
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Xiong, F., Ren, C., Mo, B. et al. A new adaptive multi-fidelity metamodel method using meta-learning and Bayesian deep learning. Struct Multidisc Optim 66, 58 (2023). https://doi.org/10.1007/s00158-023-03518-8
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1007/s00158-023-03518-8