Structural and Multidisciplinary Optimization

, Volume 59, Issue 5, pp 1621–1637 | Cite as

ESC: an efficient error-based stopping criterion for kriging-based reliability analysis methods

  • Zeyu Wang
  • Abdollah ShafieezadehEmail author
Research Paper


The ever-increasing complexity of numerical models and associated computational demands have challenged classical reliability analysis methods. Surrogate model-based reliability analysis techniques, and in particular those using kriging meta-model, have gained considerable attention recently for their ability to achieve high accuracy and computational efficiency. However, existing stopping criteria, which are used to terminate the training of surrogate models, do not directly relate to the error in estimated failure probabilities. This limitation can lead to high computational demands because of unnecessary calls to costly performance functions (e.g., involving finite element models) or potentially inaccurate estimates of failure probability due to premature termination of the training process. Here, we propose the error-based stopping criterion (ESC) to address these limitations. First, it is shown that the total number of wrong sign estimation of the performance function for candidate design samples by kriging, S, follows a Poisson binomial distribution. This finding is subsequently used to estimate the lower and upper bounds of S for a given confidence level for sets of candidate design samples classified by kriging as safe and unsafe. An upper bound of error of the estimated failure probability is subsequently derived according to the probabilistic properties of Poisson binomial distribution. The proposed upper bound is implemented in the kriging-based reliability analysis method as the stopping criterion. The efficiency and robustness of ESC are investigated here using five benchmark reliability analysis problems. Results indicate that the proposed method achieves the set accuracy target and substantially reduces the computational demand, in some cases by over 50%.


Reliability analysis Surrogate model Stopping criterion Adaptive kriging Estimation error Poisson binomial distribution 


Funding information

This research has been partly funded by the US National Science Foundation (NSF) through awards CMMI-1462183, 1563372, 1635569, and 1762918.

Compliance with ethical standards

Any opinions, findings, and conclusions or recommendations expressed in this paper are those of the authors and do not necessarily reflect the views of the National Science Foundation.


  1. Au SK, Beck JL (2003) Subset simulation and its application to seismic risk based on dynamic analysis. J Eng Mech 129(8):901–917CrossRefGoogle Scholar
  2. Balesdent M, Morio J, Marzat J (2013) Kriging-based adaptive importance sampling algorithms for rare event estimation. Struct Saf 44:1–10CrossRefzbMATHGoogle Scholar
  3. Bichon BJ, Eldred MS, Swiler LP, Mahadevan S, McFarland JM (2008) Efficient global reliability analysis for nonlinear implicit performance functions. AIAA J 46(10):2459–2468CrossRefGoogle Scholar
  4. Blatman G, Sudret B (2010) An adaptive algorithm to build up sparse polynomial chaos expansions for stochastic finite element analysis. Probabilistic Eng Mech 25(2):183–197CrossRefGoogle Scholar
  5. Bourinet J-M (2016) Rare-event probability estimation with adaptive support vector regression surrogates. Reliab Eng Syst Saf 150:210–221CrossRefGoogle Scholar
  6. Cam LL (1960) An approximation theorem for the Poisson binomial distribution. Pac J Math 10(4):1181–1197MathSciNetCrossRefzbMATHGoogle Scholar
  7. Chaudhuri A and Haftka RT, “Effectiveness indicators for stopping criteria based on minimum required improvement,” In 56th AIAA/ASCE/AHS/ASC structures, structural dynamics, and materials conference, American Institute of Aeronautics and Astronautics, 2015Google Scholar
  8. Dai H, Zhang H, Wang W, Xue G (2012) Structural reliability assessment by local approximation of limit state functions using adaptive Markov chain simulation and support vector regression. Comput-Aided Civ Infrastruct Eng 27(9):676–686CrossRefGoogle Scholar
  9. Ditlevsen O, Madsen HO (1996) Structural reliability methods, vol 178. Wiley, New YorkGoogle Scholar
  10. Dubourg V, Sudret B, Bourinet J-M (2011) Reliability-based design optimization using kriging surrogates and subset simulation. Struct Multidiscip Optim 44(5):673–690CrossRefGoogle Scholar
  11. Dubourg V, Sudret B, Deheeger F (2013) Metamodel-based importance sampling for structural reliability analysis. Probabilistic Eng Mech 33:47–57CrossRefGoogle Scholar
  12. Echard B, Gayton N, Lemaire M (2011) AK-MCS: an active learning reliability method combining kriging and Monte Carlo simulation. Struct Saf 33(2):145–154CrossRefGoogle Scholar
  13. Echard B, Gayton N, Lemaire M, Relun N (2013) A combined importance sampling and kriging reliability method for small failure probabilities with time-demanding numerical models. Reliab Eng Syst Saf 111:232–240CrossRefGoogle Scholar
  14. Fauriat W, Gayton N (2014) AK-SYS: an adaptation of the AK-MCS method for system reliability. Reliab Eng Syst Saf 123:137–144CrossRefGoogle Scholar
  15. Fishman G (2013) Monte Carlo: concepts, algorithms, and applications. Springer Science & Business MediaGoogle Scholar
  16. Gaspar B, Teixeira AP, Soares CG (2014) Assessment of the efficiency of kriging surrogate models for structural reliability analysis. Probabilistic Eng Mech 37:24–34CrossRefGoogle Scholar
  17. Gaspar B, Teixeira AP, Soares CG (2017) Adaptive surrogate model with active refinement combining kriging and a trust region method. Reliab Eng Syst SafGoogle Scholar
  18. Gayton N, Bourinet JM, Lemaire M (2003) CQ2RS: a new statistical approach to the response surface method for reliability analysis. Struct Saf 25(1):99–121CrossRefGoogle Scholar
  19. Giunta AA, McFarland JM, Swiler LP, Eldred MS (2006) The promise and peril of uncertainty quantification using response surface approximations. Struct Infrastruct Eng 2(3–4):175–189CrossRefGoogle Scholar
  20. Hu Z, Du X (2015) Mixed efficient global optimization for time-dependent reliability analysis. J Mech Des 137(5):051401–9–051401–051401Google Scholar
  21. Hu Z, Mahadevan S (2016a) A single-loop kriging (SILK) surrogate modeling for time-dependent reliability analysis. J Mech Des 138Google Scholar
  22. Hu Z, Mahadevan S (2016b) Global sensitivity analysis-enhanced surrogate (GSAS) modeling for reliability analysis. Struct Multidiscip Optim 53(3):501–521MathSciNetCrossRefGoogle Scholar
  23. Huang X, Chen J, Zhu H (2016) Assessing small failure probabilities by AK–SS: an active learning method combining kriging and subset simulation. Struct Saf 59:86–95CrossRefGoogle Scholar
  24. Johnson OT (2004) Information theory and the central limit theorem. World ScientificGoogle Scholar
  25. Kaymaz I (2005) Application of kriging method to structural reliability problems. Struct Saf 27(2):133–151CrossRefGoogle Scholar
  26. LeCam L (1965) “On the distribution of sums of independent random variables,” in Bernoulli 1713 Bayes 1763 Laplace 1813, Springer, Berlin, Heidelberg, pp. 179–202Google Scholar
  27. Lemaire M (2013) Structural reliability. John Wiley & SonsGoogle Scholar
  28. Lophaven SN, Nielsen HB, and Søndergaard J, “DACE-A Matlab Kriging toolbox, version 2.0,” 2002aGoogle Scholar
  29. Lophaven SN, Nielsen HB, and Søndergaard J, “Aspects of the matlab toolbox DACE,” Informatics and mathematical modelling, Technical University of Denmark, DTU, 2002bGoogle Scholar
  30. Lv Z, Lu Z, Wang P (2015) A new learning function for kriging and its applications to solve reliability problems in engineering. Comput Math Appl 70(5):1182–1197MathSciNetCrossRefGoogle Scholar
  31. Pedroni N, Zio E (2017) An adaptive metamodel-based subset importance sampling approach for the assessment of the functional failure probability of a thermal-hydraulic passive system. Appl Math Model 48:269–288MathSciNetCrossRefGoogle Scholar
  32. Queipo NV, Haftka RT, Shyy W, Goel T, Vaidyanathan R, Tucker PK (2005) Surrogate-based analysis and optimization. Prog Aerosp Sci 41(1):1–28Google Scholar
  33. Rajashekhar MR, Ellingwood BR (1993) A new look at the response surface approach for reliability analysis. Struct Saf 12(3):205–220CrossRefGoogle Scholar
  34. Romero VJ, Swiler LP, Giunta AA (2004) Construction of response surfaces based on progressive-lattice-sampling experimental designs with application to uncertainty propagation. Struct Saf 26(2):201–219CrossRefGoogle Scholar
  35. Roussouly N, Petitjean F, Salaun M (2013) A new adaptive response surface method for reliability analysis. Probabilistic Eng Mech 32:103–115CrossRefGoogle Scholar
  36. Rubinstein RY, Kroese DP (2016) Simulation and the Monte Carlo method. John Wiley & SonsGoogle Scholar
  37. Schueremans L, Van Gemert D (2005) Benefit of splines and neural networks in simulation based structural reliability analysis. Struct Saf 27(3):246–261CrossRefGoogle Scholar
  38. Sun Z, Wang J, Li R, Tong C (2017) LIF: a new kriging based learning function and its application to structural reliability analysis. Reliab Eng Syst Saf 157:152–165CrossRefGoogle Scholar
  39. “UQLab Kriging (Gaussian process modelling) manual,” UQLab, the framework for uncertainty quantification. [Online]. Available: [Accessed: 13-May-2017]
  40. “UQLab sensitivity analysis user manual,” UQLab, the Framework for uncertainty quantification. [Online]. Available: [Accessed: 13-May-2017]
  41. Wang Z, Shafieezadeh A (2019) REAK: reliability analysis through error rate-based adaptive kriging. Reliab Eng Syst Saf 182:33–45CrossRefGoogle Scholar
  42. Wang J, Sun Z, Yang Q, Li R (2017) Two accuracy measures of the kriging model for structural reliability analysis. Reliab Eng Syst Saf 167:494–505CrossRefGoogle Scholar
  43. Wen Z, Pei H, Liu H, Yue Z (2016) A sequential kriging reliability analysis method with characteristics of adaptive sampling regions and parallelizability. Reliab Eng Syst Saf 153:170–179CrossRefGoogle Scholar
  44. Xiao N-C, Zuo MJ, Zhou C (2018) A new adaptive sequential sampling method to construct surrogate models for efficient reliability analysis. Reliab Eng Syst Saf 169:330–338CrossRefGoogle Scholar
  45. Yang X, Liu Y, Mi C, Tang C (2018) System reliability analysis through active learning kriging model with truncated candidate region. Reliab Eng Syst Saf 169:235–241CrossRefGoogle Scholar
  46. Zhao W, Fan F, Wang W (2017) Non-linear partial least squares response surface method for structural reliability analysis. Reliab Eng Syst Saf 161:69–77CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  1. 1.The Ohio State UniversityColumbusUSA

Personalised recommendations