Estimating quantiles in imperfect simulation models using conditional density estimation

  • Michael Kohler
  • Adam KrzyżakEmail author


In this article, we consider the problem of estimating quantiles related to the outcome of experiments with a technical system given the distribution of the input together with an (imperfect) simulation model of the technical system and (few) data points from the technical system. The distribution of the outcome of the technical system is estimated in a regression model, where the distribution of the residuals is estimated on the basis of a conditional density estimate. It is shown how Monte Carlo can be used to estimate quantiles of the outcome of the technical system on the basis of the above estimates, and the rate of convergence of the quantile estimate is analyzed. Under suitable assumptions, it is shown that this rate of convergence is faster than the rate of convergence of standard estimates which ignore either the (imperfect) simulation model or the data from the technical system; hence, it is crucial to combine both kinds of information. The results are illustrated by applying the estimates to simulated and real data.


Conditional density estimation Quantile estimation Imperfect models \(L_1\) error Surrogate models Uncertainty quantification 



The authors would like to thank an anonymous referee for invaluable comments and suggestions, and they would like to thank Caroline Heil, Audrey Youmbi and Jan Benzing for pointing out an error in an early version of this manuscript. The first author would like to thank the German Research Foundation (DFG) for funding this project within the Collaborative Research Centre 805. The second author would like to acknowledge the support from the Natural Sciences and Engineering Research Council of Canada under Grant RGPIN 2015-06412.

Supplementary material

10463_2018_683_MOESM1_ESM.pdf (114 kb)
Supplementary material 1 (pdf 114 KB)


  1. Bauer, B., Devroye, L., Kohler, M., Krzy.zak, A., Walk, H. (2017). Nonparametric estimation of a function from noiseless observations at random points. Journal of Multivariate Analysis, 160, 90–104.MathSciNetCrossRefGoogle Scholar
  2. Bichon, B., Eldred, M., Swiler, M., Mahadevan, S., McFarland, J. (2008). Efficient global reliability analysis for nonlinear implicit performance functions. AIAA Journal, 46, 2459–2468.CrossRefGoogle Scholar
  3. Bott, A., Kohler, M. (2016). Adaptive estimation of a conditional density. International Statistical Review, 84, 291–316.MathSciNetCrossRefGoogle Scholar
  4. Bott, A., Kohler, M. (2017). Nonparametric estimation of a conditional density. Annals of the Institute of Statistical Mathematics, 69, 189–214.MathSciNetCrossRefGoogle Scholar
  5. Bott, A. K., Felber, T., Kohler, M. (2015). Estimation of a density in a simulation model. Journal of Nonparametric Statistics, 27, 271–285.MathSciNetCrossRefGoogle Scholar
  6. Bourinet, J.-M., Deheeger, F., Lemaire, M. (2011). Assessing small failure probabilities by combined subset simulation and support vector machines. Structural Safety, 33, 343–353.CrossRefGoogle Scholar
  7. Bucher, C., Bourgund, U. (1990). A fast and efficient response surface approach for structural reliability problems. Structural Safety, 7, 57–66.CrossRefGoogle Scholar
  8. Das, P.-K., Zheng, Y. (2000). Cumulative formation of response surface and its use in reliability analysis. Probabilistic Engineering Mechanics, 15, 309–315.CrossRefGoogle Scholar
  9. Deheeger, F., Lemaire, M. (2010). Support vector machines for efficient subset simulations: \(^{2}\)SMART method. In Proceedings of the 10th international conference on applications of statistics and probability in civil engineering (ICASP10), Tokyo, Japan.Google Scholar
  10. Devroye, L., Lugosi, G. (2001). Combinatorial methods in density estimation. New York: Springer.CrossRefGoogle Scholar
  11. Devroye, L., Felber, T., Kohler, M. (2013). Estimation of a density using real and artificial data. IEEE Transactions on Information Theory, 59(3), 1917–1928.MathSciNetCrossRefGoogle Scholar
  12. Efromovich, S. (2007). Conditional density estimation in a regression setting. Annals of Statistics, 35, 2504–2535.MathSciNetCrossRefGoogle Scholar
  13. Enss, C., Kohler, M., Krzyżak, A., Platz, R. (2016). Nonparametric quantile estimation based on surrogate models. IEEE Transactions on Information Theory, 62, 5727–5739.MathSciNetCrossRefGoogle Scholar
  14. Fan, J., Yim, T. H. (2004). A crossvalidation method for estimating conditional densities. Biometrika, 91, 819–834.MathSciNetCrossRefGoogle Scholar
  15. Fan, J., Yao, Q., Tong, H. (1996). Estimation of conditional densities and sensitivity measures in nonlinear dynamical systems. Biometrika, 83, 189–206.MathSciNetCrossRefGoogle Scholar
  16. Felber, T., Kohler, M., Krzyżak, A. (2015a). Adaptive density estimation based on real and artificial data. Journal of Nonparametric Statistics, 27, 1–18.MathSciNetCrossRefGoogle Scholar
  17. Felber, T., Kohler, M., Krzyżak, A. (2015b). Density estimation with small measurement errors. IEEE Transactions on Information Theory, 61, 3446–3456.MathSciNetCrossRefGoogle Scholar
  18. Gooijer, J. G. D., Zerom, D. (2003). On conditional density estimation. Statistica Neerlandica, 57, 159–176.MathSciNetCrossRefGoogle Scholar
  19. Györfi, L., Kohler, M., Krzyżak, A., Walk, H. (2002). A distribution-free theory of nonparametric regression. New York: Springer.CrossRefGoogle Scholar
  20. Hurtado, J. E. (2004). Structural reliability: Statistical learning perspectives. Lecture notes in applied and computational mechanics (Vol. 17). Berlin: Springer.Google Scholar
  21. Kaymaz, I. (2005). Application of Kriging method to structural reliability problems. Strutural Safety, 27, 133–151.CrossRefGoogle Scholar
  22. Kim, S.-H., Na, S.-W. (1997). Response surface method using vector projected sampling points. Structural Safety, 19, 3–19.CrossRefGoogle Scholar
  23. Kohler, M., Krzyżak, A. (2016). Estimation of a density from an imperfect simulation model (submitted).Google Scholar
  24. Kohler, M., Krzyżak, A. (2017). Improving a surrogate model in uncertainty quantification by real data (submitted).Google Scholar
  25. Kohler, M., Krzyżak, A. (2018). Adaptive estimation of quantiles in a simulation model. IEEE Transactions on Information Theory, 64, 501–512.MathSciNetCrossRefGoogle Scholar
  26. Kohler, M., Krzyżak, A., Mallapur, S., Platz, R. (2018). Uncertainty quantification in case of imperfect models: A non-Bayesian approach. Scandinavian Journal of Statistics. Scholar
  27. Mallapur, S., Platz, R. (2017). Quantification and evaluation of uncertainty in the mathematical modelling of a suspension strut using bayesian model validation approach. In Proceedings of the international modal analysis conference IMAC-XXXV, Garden Grove, California, USA, Paper 117, 30 January–2 February, 2017.Google Scholar
  28. Massart, P. (1990). The tight constant in the Dvoretzky–Kiefer–Wolfowitz inequality. Annals of Probability, 18, 1269–1283.MathSciNetCrossRefGoogle Scholar
  29. Papadrakakis, M., Lagaros, N. (2002). Reliability-based structural optimization using neural networks and Monte Carlo simulation. Computer Methods in Applied Mechanics and Engineering, 191, 3491–3507.CrossRefGoogle Scholar
  30. Parzen, E. (1962). On the estimation of a probability density function and the mode. Annals of Mathematical Statistics, 33, 1065–1076.MathSciNetCrossRefGoogle Scholar
  31. Rosenblatt, M. (1956). Remarks on some nonparametric estimates of a density function. Annals of Mathematical Statistics, 27, 832–837.MathSciNetCrossRefGoogle Scholar
  32. Rosenblatt, M. (1969). Conditional probability density and regression estimates. In P. R. Krishnaiah (Ed.), Multivariate analysis II (pp. 25–31). New York: Academic Press.Google Scholar
  33. Wong, R. K. W., Storlie, C. B., Lee, T. C. M. (2017). A frequentist approach to computer model calibration. Journal of the Royal Statistical Society, Series B, 79, 635–648.MathSciNetCrossRefGoogle Scholar

Copyright information

© The Institute of Statistical Mathematics, Tokyo 2018

Authors and Affiliations

  1. 1.Fachbereich MathematikTechnische Universität DarmstadtDarmstadtGermany
  2. 2.Department of Computer Science and Software EngineeringConcordia UniversityMontrealCanada

Personalised recommendations