Skip to main content
SpringerLink
Log in

Sequential design of computer experiments for the estimation of a probability of failure

  • Published:
Statistics and Computing Aims and scope Submit manuscript

Abstract

This paper deals with the problem of estimating the volume of the excursion set of a function f:ℝd→ℝ above a given threshold, under a probability measure on ℝd that is assumed to be known. In the industrial world, this corresponds to the problem of estimating a probability of failure of a system. When only an expensive-to-simulate model of the system is available, the budget for simulations is usually severely limited and therefore classical Monte Carlo methods ought to be avoided. One of the main contributions of this article is to derive SUR (stepwise uncertainty reduction) strategies from a Bayesian formulation of the problem of estimating a probability of failure. These sequential strategies use a Gaussian process model of f and aim at performing evaluations of f as efficiently as possible to infer the value of the probability of failure. We compare these strategies to other strategies also based on a Gaussian process model for estimating a probability of failure.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Arnaud, A., Bect, J., Couplet, M., Pasanisi, A., Vazquez, E.: Évaluation d’un risque d’inondation fluviale par planification séquentielle d’expériences. In: 42èmes Journées de Statistique (2010)

    Google Scholar 

  • Au, S.K., Beck, J.: Estimation of small failure probabilities in high dimensions by subset simulation. Probab. Eng. Mech. 16(4), 263–277 (2001)

    Article  Google Scholar 

  • Bayarri, M.J., Berger, J.O., Paulo, R., Sacks, J., Cafeo, J.A., Cavendish, J., Lin, C.H., Tu, J.: A framework for validation of computer models. Technometrics 49(2), 138–154 (2007)

    Article  MathSciNet  Google Scholar 

  • Berry, D.A., Fristedt, B.: Bandit Problems: Sequential Allocation of Experiments. Chapman & Hall, London (1985)

    MATH  Google Scholar 

  • Bertsekas, D.P.: Dynamic Programming and Optimal Control, vol. 1. Athena Scientific, Nashua (1995)

    MATH  Google Scholar 

  • Bichon, B.J., Eldred, M.S., Swiler, L.P., Mahadevan, S., McFarland, J.M.: Efficient global reliability analysis for nonlinear implicit performance functions. AIAA J. 46(10), 2459–2468 (2008)

    Article  Google Scholar 

  • Bjerager, P.: On computational methods for structural reliability analysis. Struct. Saf. 9, 76–96 (1990)

    Article  Google Scholar 

  • Borri, A., Speranzini, E.: Structural reliability analysis using a standard deterministic finite element code. Struct. Saf. 19(4), 361–382 (1997)

    Article  Google Scholar 

  • Box, G.E.P., Draper, N.R.: Empirical Model-Building and Response Surfaces. Wiley, New York (1987)

    MATH  Google Scholar 

  • Bucher, C.G., Bourgund, U.: A fast and efficient response surface approach for structural reliability problems. Struct. Saf. 7(1), 57–66 (1990)

    Article  Google Scholar 

  • Chilès, J.P., Delfiner, P.: Geostatistics: Modeling Spatial Uncertainty. Wiley, New York (1999)

    Book  MATH  Google Scholar 

  • Currin, C., Mitchell, T., Morris, M., Ylvisaker, D.: Bayesian prediction of deterministic functions, with applications to the design and analysis of computer experiments. J. Am. Stat. Assoc. 86(416), 953–963 (1991)

    Article  MathSciNet  Google Scholar 

  • Deheeger, F.: Couplage mécano-fiabiliste : 2SMART—méthodologie d’apprentissage stochastique en fiabilité. Ph.D. thesis, Université Blaise Pascal, Clermont II (2008)

  • Deheeger, F., Lemaire, M.: Support vector machine for efficient subset simulations: 2SMART method. In: Kanda, J., Takada, T., Furuta, H. (eds.) 10th International Conference on Application of Statistics and Probability in Civil Engineering, Proceedings and Monographs in Engineering, Water and Earth Sciences, pp. 259–260. Taylor & Francis, London (2007)

    Google Scholar 

  • Echard, B., Gayton, N., Lemaire, M.: Kriging-based Monte Carlo simulation to compute the probability of failure efficiently: AK-MCS method. In: 6èmes Journées Nationales de Fiabilité, Toulouse, France, 24–26 mars 2010a

    Google Scholar 

  • Echard, B., Gayton, N., Lemaire, M.: Structural reliability assessment using kriging metamodel and active learning. In: IFIP WG 7.5 Working Conference on Reliability and Optimization of Structural Systems (2010b)

    Google Scholar 

  • Fleuret, F., Geman, D.: Graded learning for object detection. In: Proceedings of the Workshop on Statistical and Computational Theories of Vision of the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR/SCTV) (1999)

    Google Scholar 

  • Frazier, P.I., Powell, W.B., Dayanik, S.: A knowledge-gradient policy for sequential information collection. SIAM J. Control Optim. 47(5), 2410–2439 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  • Ginsbourger, D.: Métamodèles multiples pour l’approximation et l’optimisation de fonctions numériques multivariables. Ph.D. thesis, Ecole nationale supérieure des Mines de Saint-Etienne (2009)

  • Ginsbourger, D., Le Riche, R., Carraro, L.: Kriging is well-suited to parallelize optimization. In: Hiot, L.M., Ong, Y.S., Tenne, Y., Goh, C.K. (eds.) Computational Intelligence in Expensive Optimization Problems. Adaptation Learning and Optimization, vol. 2, pp. 131–162. Springer, Berlin (2010)

    Chapter  Google Scholar 

  • Handcock, M.S., Stein, M.L.: A Bayesian analysis of kriging. Technometrics 35(4), 403–410 (1993)

    Article  Google Scholar 

  • Hornik, K.: The R FAQ (2010). http://CRAN.R-project.org/doc/FAQ/R-FAQ.html. ISBN 3-900051-08-9

  • Hurtado, J.E.: An examination of methods for approximating implicit limit state functions from the viewpoint of statistical learning theory. Struct. Saf. 26(3), 271–293 (2004)

    Article  Google Scholar 

  • Hurtado, J.E.: Filtered importance sampling with support vector margin: a powerful method for structural reliability analysis. Struct. Saf. 29(1), 2–15 (2007)

    Article  Google Scholar 

  • Jones, D.R., Schonlau, M., William, J.: Efficient global optimization of expensive black-box functions. J. Glob. Optim. 13(4), 455–492 (1998)

    Article  MATH  Google Scholar 

  • Kennedy, M., O’Hagan, A.: Bayesian calibration of computer models. J. R. Stat. Soc., Ser. B, Stat. Methodol. 63(3), 425–464 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  • Kimeldorf, G.S., Wahba, G.: A correspondence between Bayesian estimation on stochastic processes and smoothing by splines. Ann. Math. Stat. 41(2), 495–502 (1970)

    Article  MathSciNet  MATH  Google Scholar 

  • Kushner, H.J.: A new method of locating the maximum point of an arbitrary multipeak curve in the presence of noise. J. Basic Eng. 86, 97–106 (1964)

    Article  Google Scholar 

  • Loeppky, J.L., Sacks, J., Welch, W.J.: Choosing the sample size of a computer experiment: a practical guide. Technometrics 51(4), 366–376 (2009)

    Article  MathSciNet  Google Scholar 

  • Mockus, J.: Bayesian Approach to Global Optimization. Theory and Applications. Kluwer Academic, Dordrecht (1989)

    Book  MATH  Google Scholar 

  • Mockus, J., Tiesis, V., Zilinskas, A.: The application of Bayesian methods for seeking the extremum. In: Dixon, L., Szego, E.G. (eds.) Towards Global Optimization, vol. 2, pp. 117–129. Elsevier, Amsterdam (1978)

    Google Scholar 

  • Oakley, J.: Estimating percentiles of uncertain computer code outputs. J. R. Stat. Soc., Ser. C 53(1), 83–93 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  • Oakley, J., O’Hagan, A.: Bayesian inference for the uncertainty distribution of computer model outputs. Biometrika 89(4) (2002)

  • Oakley, J., O’Hagan, A.: Probabilistic sensitivity analysis of complex models: a Bayesian approach. J. R. Stat. Soc., Ser. B, Stat. Methodol. 66(3), 751–769 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  • O’Hagan, A.: Curve fitting and optimal design for prediction. J. R. Stat. Soc., Ser. B, Methodol. 40(1), 1–42 (1978)

    MathSciNet  MATH  Google Scholar 

  • Parzen, E.: An approach to time series analysis. Ann. Math. Stat. 32, 951–989 (1962)

    Article  MathSciNet  Google Scholar 

  • Paulo, R.: Default priors for Gaussian processes. Ann. Stat. 33(2), 556–582 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  • Picheny, V., Ginsbourger, D.: KrigInv: Kriging-based inversion for deterministic and noisy computer experiments, version 1.1 (2011). http://cran.r-project.org/web/packages/KrigInv

  • Picheny, V., Ginsbourger, D., Roustant, O., Haftka, R.T., Kim, N.H.: Adaptive designs of experiments for accurate approximation of target regions. J. Mech. Des. 132(7), 1–9 (2010)

    Article  Google Scholar 

  • Piera-Martinez, M.: Modélisation des comportements extrêmes en ingénierie. Ph.D. thesis, Université Paris Sud, Paris XI (2008)

  • Pradlwarter, H., Schuëller, G., Koutsourelakis, P., Charmpis, D.: Application of line sampling simulation method to reliability benchmark problems. Struct. Saf. 29(3), 208–221 (2007). A Benchmark Study on Reliability in High Dimensions

    Article  Google Scholar 

  • Press, W.H., Teukolsky, S.A., Vetterling, W.T., Flannery, B.P.: Numerical Recipes in C. The Art of Scientific Computing, 2nd edn. Cambridge University Press, Cambridge (1992)

    MATH  Google Scholar 

  • Rajashekhar, M.R., Ellingwood, B.R.: A new look at the response surface approach for reliability analysis. Struct. Saf. 12(3), 205–220 (1993)

    Article  Google Scholar 

  • Ranjan, P., Bingham, D., Michailidis, G.: Sequential experiment design for contour estimation from complex computer codes. Technometrics 50(4), 527–541 (2008)

    Article  MathSciNet  Google Scholar 

  • Rubinstein, R., Kroese, D.: The Cross-Entropy Method. Springer, Berlin (2004)

    MATH  Google Scholar 

  • Sacks, J., Welch, W.J., Mitchell, T.J., Wynn, H.P.: Design and analysis of computer experiments. Stat. Sci. 4(4), 409–435 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  • Santner, T.J., Williams, B.J., Notz, W.: The Design and Analysis of Computer Experiments. Springer, Berlin (2003)

    MATH  Google Scholar 

  • Schueremans, L.: Probabilistic evaluation of structural unreinforced masonry. Ph.D. thesis, Catholic University of Leuven (2001)

  • Schueremans, L., Gemert, D.V.: Benefit of splines and neural networks in simulation based structural reliability analysis. Struct. Saf. 27(3), 246–261 (2005)

    Article  Google Scholar 

  • Stein, M.L.: Interpolation of Spatial Data: Some Theory for Kriging. Springer, New York (1999)

    Book  MATH  Google Scholar 

  • Vazquez, E., Bect, J.: A sequential Bayesian algorithm to estimate a probability of failure. In: Proceedings of the 15th IFAC Symposium on System Identification, SYSID 2009, Saint-Malo, France (2009)

    Google Scholar 

  • Vazquez, E., Piera-Martinez, M.: Estimation du volume des ensembles d’excursion d’un processus gaussien par krigeage intrinsèque. In: 39ème Journées de Statistiques Conférence Journée de Statistiques, Angers, France (2007)

    Google Scholar 

  • Vestrup, E.M.: The Theory of Measures and Integration. Wiley, New York (2003)

    Book  MATH  Google Scholar 

  • Villemonteix, J.: Optimisation de fonctions coûteuses. Ph.D. thesis, Université Paris-Sud XI, Faculté des Sciences d’Orsay (2008)

  • Villemonteix, J., Vazquez, E., Walter, E.: An informational approach to the global optimization of expensive-to-evaluate functions. J. Glob. Optim. 44(4), 509–534 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  • Waarts, P.H.: Structural reliability using finite element methods: an appraisal of DARS. Ph.D. thesis, Delft University of Technology (2000)

  • Welch, W.J., Buck, R.J., Sacks, J., Wynn, H.P., Mitchell, T.J., Morris, M.D.: Screening, predicting and computer experiments. Technometrics 34, 15–25 (1992)

    Article  Google Scholar 

  • Yaglom, A.M.: Correlation Theory of Stationary and Related Random Functions I: Basic Results. Springer Series in Statistics. Springer, New York (1986)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. SUPELEC, Gif-sur-Yvette, France

    Julien Bect, Ling Li & Emmanuel Vazquez

  2. Ecole Centrale Paris, Chatenay-Malabry, France

    Victor Picheny

  3. Institute of Mathematical Statistics and Actuarial Science, University of Bern, Bern, Switzerland

    David Ginsbourger

Authors
  1. Julien Bect
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. David Ginsbourger
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ling Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Victor Picheny
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Emmanuel Vazquez
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Julien Bect or Emmanuel Vazquez.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Bect, J., Ginsbourger, D., Li, L. et al. Sequential design of computer experiments for the estimation of a probability of failure. Stat Comput 22, 773–793 (2012). https://doi.org/10.1007/s11222-011-9241-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11222-011-9241-4

Keywords

Search

Navigation