Skip to main content
SpringerLink
Log in

A bayesian model calibration under insufficient data environment

  • Research Paper
  • Published:
Structural and Multidisciplinary Optimization Aims and scope Submit manuscript

Abstract

In recent years, remarkable advances in computing performance and computer-aided engineering have enabled reliability-based design optimization (RBDO) to guarantee the target reliability of a product. For successful product development through RBDO, it is indispensable to clarify uncertainties of unknown model variables. In most cases, however, due to cost and time constraints, there are not enough test data, which can lead to a less reliable optimum. For this reason, the primary purpose of this study is to propose a pragmatic approach to perform an inverse uncertainty quantification or a statistical model calibration more accurately and efficiently under an insufficient data environment. Based on the Bayesian model calibration framework, the proposed method consists of two main steps: (1) prior distribution prediction using output (i.e., component) test data and (2) posterior distribution prediction using input (i.e., coupon) test data. In the prior distribution prediction step, the maximum likelihood estimate (MLE) is used to obtain the estimated statistical parameters, the distribution type of unknown model variables, and the Fisher information matrix (FIM) to calculate variances of the estimated statistical parameters. The posterior distribution prediction step utilizes the Bayes’ theorem, which combines the prior distribution with the likelihood obtained by reflecting the input test data into the probability density of the estimated unknown model variable. During this process, each test data that is insufficient to directly model or indirectly predict the probability density of the unknown model variable can be integrated to address the crucial issue of the insufficient data effectively. Mathematical and engineering examples are utilized to validate the proposed method for quantification of unknown model variables.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23

Similar content being viewed by others

Abbreviations

\({\mathbf{d}}\) :

A controllable design variable vector

\(z^{{\text{s}}} \left( \cdot \right)\) :

Response function of a simulation model

\({{\varvec{\upxi}}}\) :

A known model variable vector

\(\varphi \left( \cdot \right)\) :

Function as a calibration metric

\({{\varvec{\uptheta}}}\) :

An unknown model variable vector

\(L\left( \cdot \right)\) :

Likelihood function

\({{\varvec{\Theta}}}\) :

A vector of the statistical parameters of the unknown model variables

\(l\left( \cdot \right)\) :

Log-likelihood function

\(\zeta\) :

Distribution type of the unknown model variable

\(p\left( \cdot \right)\) :

Posterior PDF

\(\delta \left( \cdot \right)\) :

A discrepancy function between the experimental and the simulation model

\(\pi \left( \cdot \right)\) :

Prior PDF

\(\varepsilon\) :

A measurement error

\(\hat{F}_{{x_{i} }} \left( \cdot \right)\) :

CDF of an unknown model variable at calibration site \(x_{i}\)

\({\mathbf{y}}^{{\text{e}}}\) :

An output test data vector

\(U_{{a,m_{v} }}^{{\text{m}}} \left( \cdot \right)\) :

PDF of area-metric calculated by taking \(m_{v}\) observations from the estimated probability model

\({\mathbf{x}}^{{\text{e}}}\) :

An input test data vector

\(T_{{n_{v} }} \left( \alpha \right)\) :

Threshold of \(\alpha\) significance level for \(n_{v}\) validation samples

\(y_{i}^{{\text{e}}} {, }y_{j}^{{\text{e}}}\) :

Output test data for calibration and validation, respectively

\(n_{c} {, }n_{v}\) :

The quantity of output observations for calibration and validation, respectively

\(x_{i}^{{\text{e}}} {, }x_{j}^{{\text{e}}}\) :

Input test data for calibration and validation, respectively

\(m_{c} {, }m_{v}\) :

The quantity of input observations for calibration and validation, respectively

\({\overline{\mathbf{I}}}\) :

Expected Fisher information matrix in entire observations

\(k\) :

Number of unknown model variables

\({\mathbf{I}}\) :

Observed Fisher information matrix in entire observations

\(u_{j}\) :

The u-value; CDF value at validation site \(x_{j}\)

\({\mathbf{I}}_{1}\) :

The Fisher information matrix in a single observation

\(u_{{a,m_{v} }}^{{{(}j{)}}}\) :

\(j\)-th area-metric calculated by taking \(m_{v}\) observations from the estimated probability model

\(a{, }b\) :

The model parameters of the given parametric PDF

\(u_{{a,m_{v} }}^{{\text{m}}}\) :

Area-metric value calculated by taking \(m_{v}\) observations from the estimation

\(z^{{\text{e}}} \left( \cdot \right)\) :

Response function of experiment

\(u_{{a,m_{v} }}^{{\text{e}}}\) :

Area-metric value calculated by taking \(m_{v}\) observations from experiment

\(F_{X} {, }F_{Y}\) :

Horizontal and vertical loads acting on a cantilever beam, respectively

\(\lambda_{1} {,}\lambda_{2} {,}\lambda_{3}\) :

Principal stretches

\(w{, }t\) :

Width and thickness of the cantilever beam, respectively

\(I_{1} {,}I_{2} {,}I_{3}\) :

Strain invariants

\(D\) :

The deflections of the cantilever beam

\(s\) :

Strain energy

\(E\) :

The elastic modulus of the cantilever beam

\(W\left( \cdot \right)\) :

Function of strain energy potential

\(L\) :

The length of the cantilever beam

\(C_{10} {, }B_{1}\) :

The material parameters of the Neo-Hookean function

References

  • ABAQUS Documentation. (2014) Dassault Systèmes Simulia Corp

  • Arendt PD, Apley DW, Chen W (2012a) Quantification of model uncertainty: Calibration, model discrepancy, and identifiability. J Mech Des 134(10):7390

    Google Scholar 

  • Arendt PD, Apley DW, Chen W, Lamb D, Gorsich D (2012b) Improving identifiability in model calibration using multiple responses. J Mech Des 10:7573

    Google Scholar 

  • Bi Z (2018) Finite element analysis applications: A Systematic and practical approach. Academic Press, Cambridge

    MATH  Google Scholar 

  • Bishop CM (2006) Pattern recognition and machine learning. Springer

    MATH  Google Scholar 

  • Blatman G, Sudret B (2010) An adaptive algorithm to build up sparse polynomial chaos expansion for stochastic finite element analysis. Probabilistic Eng Mech 25:183–197

    Google Scholar 

  • Campbell K (2006) Statistical calibration of computer simulations. Reliab Eng Syst Saf 91(10–11):1358–1363

    Google Scholar 

  • Cavanaugh JE, Shumway RH (1996) On computing the expected Fisher information matrix for state-space model parameters. Stat Probab Lett 26:347–355

    MathSciNet  MATH  Google Scholar 

  • DeGroot MH, Schervish MJ (2011) Probability and statistics. Pearson Education. Hoboken

  • Deisenroth MP, Faisal AA, Ong CS (2020) Mathematics for machine learning. Cambridge University Press

    MATH  Google Scholar 

  • Efron B, Hinkley DV (1978) Assessing the accuracy of the maximum likelihood estimator: Observed versus expected Fisher information. Biometrika 65(3):457–487

    MathSciNet  MATH  Google Scholar 

  • Fahrmeir L, Kaufmann H (1985) Consistency and asymptotic normality of the maximum likelihood estimator in generalized linear models. Ann Stat 13(1):342–368

    MathSciNet  MATH  Google Scholar 

  • Ferson S, Oberkampf WL (2009) Validation of imprecise probability models. Int J Reliab Saf 3(1):3–22

    Google Scholar 

  • Ferson S, Oberkampf WL, Ginzburg L (2008) Model validation and predictive capability for the thermal challenge problem. Comput Methods Appl Mech Eng 197(29–32):2408–2430

    MATH  Google Scholar 

  • Hess PE, Bruchman D, Assakkaf IA, Ayyub BM (2002) Uncertainties in material and geometric strength and load variables. Nav Eng J 114(2):139–166

    Google Scholar 

  • Higdon D, Nakhleh C, Gattiker J, Williams B (2008) A Bayesian calibration approach to the thermal problem. Comput Methods Appl Mech Eng 197(29–32):2431–2441

    MATH  Google Scholar 

  • Honarmandi P, Arroyave R (2020) Uncertainty quantification and propagation in computational materials science and simulation-assisted materials. Integr Mater Manuf Innov 9:103–143

    Google Scholar 

  • Hossain M, Steinmann P (2013) More hyperelastic models for rubber-like materials: consistent tangent operators and comparative study. J Mech Behav Mater 22(1–2):27–50

    Google Scholar 

  • Hu J, Zhou Q, McKeand A, Xie T, Choi SK (2021) A model validation framework based on parameter calibration under aleatory and epistemic uncertainty. Struct Multidisc Optim 63:645–660

    Google Scholar 

  • Jiang C, Hu Z, Liu Y, Mourelatos ZP, Gorsich D, Jayakumar P (2020) A sequential calibration and validation framework for model uncertainty quantification and reduction. Comput Methods Appl Mech Eng 368:113172

    MathSciNet  MATH  Google Scholar 

  • Jung BC, Park J, Oh H, Kim J, Youn BD (2015) A framework of model validation and virtual product qualification with limited experimental data based on statistical inference. Struct Multidisc Optim 51:573–583

    Google Scholar 

  • Jung Y, Kang K, Cho H, Lee I (2021) Confidence-based design optimization for a more conservative optimum under surrogate model uncertainty caused by Gaussian process. J Mech Des 143(9):091701

    Google Scholar 

  • Kalinina A, Spada M, Vetsch DF, Marelli S, Whealton C, Burgherr P, Sudret B (2020) Metamodeling for uncertainty quantification of a flood wave model for concrete dam breaks. Energies 13(14):3685

    Google Scholar 

  • Kang YJ, Noh Y, Lim OK (2018) Kernel density estimation with bounded data. Struct Multidisc Optim 57:95–113

    MathSciNet  Google Scholar 

  • Kang K, Qin C, Lee BJ, Lee I (2019) Modified screening-based Kriging method with cross validation and application to engineering design. Appl Math Model 70:626–642

    MathSciNet  MATH  Google Scholar 

  • Kennedy MC, O’Hagan A (2001) Bayesian calibration of computer models. J R Stat Soc Ser B Methodol 63(3):425–464

    MathSciNet  MATH  Google Scholar 

  • Khalil M, Teichert GH, Alleman C, Heckman NM, Jones RE, Garikipati K, Boyce BL (2021) Modeling strength and failure variability due to porosity in additively manufactured metals. Comput Methods Appl Mech Eng 373:113471

    MathSciNet  MATH  Google Scholar 

  • Kim S, Shin H, Rhim S, Rhee KY (2019) Calibration of hyperelastic and hyperfoam constitutive models for an indentation event of rigid polyurethane foam. Compos B Eng 163:297–302

    Google Scholar 

  • Lee G, Kim W, Oh H, Youn BD, Kim NH (2019a) Review of statistical model calibration and validation—from the perspective of uncertainty structures. Struct Multidisc Optim 60:1619–1644

    MathSciNet  Google Scholar 

  • Lee G, Son H, Youn BD (2019b) Sequential optimization and uncertainty propagation method for efficient optimization-based model calibration. Struct Multidisc Optim 60:1355–1372

    Google Scholar 

  • Li M, Wang Z (2018) Confidence-driven design optimization using Gaussian process metamodeling with insufficient data. J Mech Des 140(12):121405

    Google Scholar 

  • Li W, Chen W, Jiang Z, Lu Z, Liu Y (2014) New validation metrics for models with multiple correlated responses. Reliab Eng Syst Saf 127:1–11

    Google Scholar 

  • Lim W, Lee TH, Kang S, Cho S (2016) Estimation of body and tail distribution under extreme events for reliability analysis. Struct Multidisc Optim 54:1631–1639

    MathSciNet  Google Scholar 

  • Linnet K (2000) Nonparametric estimation of reference intervals by simple and bootstrap-based procedure. Clin Chem 46:867–869

    Google Scholar 

  • Liu Y, Chen W, Arendt P, Huang HZ (2011) Toward a better understanding of model validation metrics. J Mech Des 133(7):071005

    Google Scholar 

  • Ly A, Marsman M, Verhagen J, Grasman R, Wagenmakers E (2017) A Tutorial on Fisher information. J Math Psychol 80:40–55

    MathSciNet  MATH  Google Scholar 

  • Marckmann G, Verron E (2006) Comparison of hyperelastic models for rubber-like materials. Rubber Chem Technol 79(5):835–858

    Google Scholar 

  • McFarland J, Mahadevan S (2008) Calibration and uncertainty analysis for computer simulations with multivariate output. AIAA J 46(5):1253–1265

    Google Scholar 

  • MIL-HDBK-5H (1998) Metallic Material and Elements for Aerospace Vehicle Structures. Tech. Rep. MIL-HDBK-5H, U.S. Department of Defense

  • Moon M, Choi KK, Cho H, Gaul N, Lamb D, Gorsich D (2017) Reliability-based design optimization using confidence-based model validation for insufficient experimental data. J Mech Des 139(3):031404

    Google Scholar 

  • Moon M, Choi KK, Gaul N, Lamb D (2019) Treating epistemic uncertainty using bootstrapping selection of input distribution model for confidence-based reliability assessment. J Mech Des 141(3):031402

    Google Scholar 

  • Moreira DC, Nunes NCS (2013) Comparison of simple and pure shear for an incompressible isotropic hyperelastic material under large deformation. Polym Test 32(2):240–248

    Google Scholar 

  • Myung JI, Navarro DJ (2005) Information matrix. Encyclopedia of Statistics in Behavioral Science 2:923–924

    Google Scholar 

  • Nah JS, Lee J (2020) Reliability assessment of display delamination considering adhesive properties based on statistical model calibration and validation. Int J Mech Mater Des 16:191–206

    Google Scholar 

  • Oh H, Wei H, Han B, Youn BD (2016) Probabilistic lifetime prediction of electronic packages using advanced uncertainty propagation analysis and model calibration. IEEE Trans Compon Packaging Manuf Technol 6(2):238–248

    Google Scholar 

  • Oh H, Choi H, Jung JH, Yoon BD (2019) A robust and convex metric for unconstrained optimization in statistical model calibration-probability residual (PR). Struct Multidisc Optim 60:1171–1187

    Google Scholar 

  • Pashazadeh S, Sharifi M (2008) Reliability assessment under uncertainty using Dempster-Shafer and vague set theory. IEEE Int Conf Comput Intell Meas Syst Appl, Istanbul, Turkey, July 14–16

  • Picheny V, Kim NH, Haftka RT (2009) Application of bootstrap method in conservative estimation of reliability with limited samples. Struct Multidisc Optim 41:205–217

    MathSciNet  MATH  Google Scholar 

  • Prokopenko M, Lizier JT, Obst O, Wang R (2011) Relating Fisher information to order parameters. Phys Rev E 84:041116

    Google Scholar 

  • Ralph CS (2014) Uncertainty quantification: Theory, implementation, and applications. Society for Industrial and Applied Mathematics, Philadelphia

    MATH  Google Scholar 

  • Rao SS, Annamdas KK (2008) Evidence-based fuzzy approach for the safety analysis of uncertain systems. AIAA J 46(9):2383–2387

    Google Scholar 

  • Romanov KI (2001) The drucker stability of a material. J Appl Math Mech 65(1):15–162

    MATH  Google Scholar 

  • Roy CJ, Oberkampf WL (2011) A Comprehensive framework for verification, validation, and uncertainty quantification in scientific computing. Comput Methods Appl Mech Eng 200:2131–2244

    MathSciNet  MATH  Google Scholar 

  • Sargsyan K, Najm HN, Ghanem R (2015) On the statistical calibration of physical models. Int J Chem Kinet 47(4):246–276

    Google Scholar 

  • Son H, Lee G, Kang K, Kang Y, Youn BD, Lee I, Noh Y (2020) Industrial issues and solutions to statistical model improvement: a case study of an automobile steering column. Struct Multidisc Optim 60:1739–1756

    Google Scholar 

  • Sourati J, Akcakaya M, Leen TK, Erdogmus D, Dy JG (2017) Asymptotic analysis of objectives based on Fisher information in active learning. J Mach Learn Res 18:1–41

    MathSciNet  MATH  Google Scholar 

  • Steinmann P, Hossain M, Possart G (2012) Hyperelastic models for rubber-like materials: consistent tangent operators and suitability for Treloar’s data. Arch Appl Mech 82:1183–1217

    MATH  Google Scholar 

  • Sun NZ, Sun A (2015) Model calibration and parameter estimation: For environmental and water resource systems. Springer-Verlag, New York

    MATH  Google Scholar 

  • Trucano TG, Swiler LP, Igusa T, Oberkampf WL, Pilch M (2006) Calibration, validation, and sensitivity analysis: What’s what. Reliab Eng Syst Saf 91(10–11):1331–1357

    Google Scholar 

  • Vakilzadeh MK, Yaghoubi V, Johansson AT, Abrahamsson TJS (2017) Stochastic finite element model calibration based on frequency responses and bootstrap sampling. Mech Syst Signal Process 88:180–198

    Google Scholar 

  • Viana FAC, Haftka RT, Watson LT (2013) Efficient global optimization algorithm assisted by multiple surrogate technique. J Glob Optim 56:669–689

    MATH  Google Scholar 

  • Villaverde AF, Fröhlich F, Weindl D, Hasenauer J, Banga JR (2019) Benchmarking optimization methods for parameter estimation in large kinetic models. Bioinformatics 35(5):830–838

    Google Scholar 

  • Vrugt JA (2016) Markov chain Monte Carlo simulation using the DREAM software package: Theory, concepts, and MATLAB implementation. Environ Model Softw 75:273–316

    Google Scholar 

  • Wachter E, Ngu TQ, Alirand M (2019) Virtual simulation of an electro-hydraulic braking system. ATZ Worldwide 121:54–59. https://doi.org/10.1007/s38311-019-0070-y

    Article  Google Scholar 

  • Wehrens R, Putter H, Buydens LMC (2000) The bootstrap: a tutorial. Chemom Intell Lab Syst 54:35–52

    Google Scholar 

  • Wu YT, Shin Y, Sues R, Cesare M (2001) Safety-factor based approach for probability-based design optimization. In: Proceedings of the 42nd AIAA/ASME/ASCE/AHS/ASC Struct Struct Dyn Mater Conf, number AIAA-2001–1522, Seattle, WA, 196, 199–342

  • Xi Z (2019) Model-based reliability analysis with both model uncertainty and parameter uncertainty. J Mech Des 141(5):051404

    Google Scholar 

  • Xi Z, Jung BC, Youn BD (2012) Random field modeling with insufficient data sets for probability analysis. In: Proc Annu Reliab Maintainab Symp, Reno, NV, USA, January 23–26

  • Xiong Y, Chen W, Tsui K, Apley DW (2009) A better understanding of model updating strategies in validating engineering models. Comput Methods Appl Mech Eng 198:1327–1337

    MATH  Google Scholar 

  • Youn BD, Jung BC, Xi Z, Kim SB, Lee WR (2011) A hierarchical framework for statistical model calibration in engineering product development. Comput Methods Appl Mech Eng 200:1421–1431

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mechanical Engineering, Korea Advanced Institute of Science and Technology, Daejeon, 34141, South Korea

    Jeonghwan Choo, Yongsu Jung & Ikjin Lee

  2. CAE, Global Brake BU, Mando Corporation, Gyeonggi-do, Seongnam, 13486, South Korea

    Jeonghwan Choo

Authors
  1. Jeonghwan Choo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yongsu Jung
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ikjin Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ikjin Lee.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Replication of results

The results presented in this paper are obtained from our in-house MATLAB codes. For academic use only, contact the corresponding authors to get a copy of the source codes.

Additional information

Responsible Editor: Chao Hu

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Choo, J., Jung, Y. & Lee, I. A bayesian model calibration under insufficient data environment. Struct Multidisc Optim 65, 96 (2022). https://doi.org/10.1007/s00158-022-03196-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00158-022-03196-y

Keywords

Search

Navigation