pp 1–29 | Cite as

Nonparametric quantile estimation using surrogate models and importance sampling

  • Michael KohlerEmail author
  • Reinhard Tent


Given a costly to compute function \(m: {\mathbb {R}}^d\rightarrow {\mathbb {R}}\), which is part of a simulation model, and an \({\mathbb {R}}^d\)-valued random variable with known distribution, the problem of estimating a quantile \(q_{m(X),\alpha }\) is investigated. The presented approach has a nonparametric nature. Monte Carlo quantile estimates are obtained by estimating m through some estimate (surrogate) \(m_n\) and then by using an initial quantile estimate together with importance sampling to construct an importance sampling surrogate quantile estimate. A general error bound on the error of this quantile estimate is derived, which depends on the local error of the function estimate \(m_n\), and the convergence rates of the corresponding importance sampling surrogate quantile estimates are analyzed. The finite sample size behavior of the estimates is investigated by applying the estimates to simulated data.


Nonparametric quantile estimation Importance sampling Surrogate models Rate of convergence 

Mathematics Subject Classification

Primary 62G05 Secondary 62G30 



The authors would like to thank the German Research Foundation (DFG) for funding this project within the Collaborative Research Centre 805 (Projektnummer 57157408 - SFB 805). Furthermore the authors would like to thank two anonymous referees and the Associate editor for many very useful comments which improved an early version of this article.

Compliance with ethical standards

Conflict of interest

On behalf of all authors, the corresponding author states that there is no conflict of interest.


  1. Arnold BC, Balakrishnan N, Nagaraja HN (1992) A first course in order statistics. Wiley, New YorkzbMATHGoogle Scholar
  2. Beirlant J, Györfi L (1998) On the asymptotic \({L}_2\)-error in partitioning regression estimation. J Stat Plan Inference 71:93–107CrossRefzbMATHGoogle Scholar
  3. Bichon B, Eldred M, Swiler M, Mahadevan S, McFarland J (2008) Efficient global reliability analysis for nonlinear implicit performance functions. AIAA J 46:2459–2468CrossRefGoogle Scholar
  4. Bourinte J-M, Deheeger F, Lemaire M (2011) Assessing small failure probabilities by combined subset simulation and support vector machines. Struct Saf 33:343–353CrossRefGoogle Scholar
  5. Bucher C, Bourgund U (1990) A fast and efficient response surface approach for structural reliability problems. Struct Saf 7:57–66CrossRefGoogle Scholar
  6. Cannamela C, Garnier J, Iooss B (2008) Controlled stratification for quantile estimation. Ann Appl Stat 2(4):1554–1580MathSciNetCrossRefzbMATHGoogle Scholar
  7. Das P-K, Zheng Y (2000) Cumulative formation of response surface and its use in reliability analysis. Probab Eng Mech 15:309–315CrossRefGoogle Scholar
  8. de Boor C (1978) A practical guide to splines. Springer, BerlinCrossRefzbMATHGoogle Scholar
  9. Deheeger F, Lemaire M (2010) Support vector machines for efficient subset simulations: \(^2\)SMART method. In: Proceedings of the 10th international conference on applications of statistics and probability in civil engineering (ICASP10), Tokyo, JapanGoogle Scholar
  10. Devroye L (1982) Necessary and sufficient conditions for the almost everywhere convergence of nearest neighbor regression function estimates. Zeitschrift für Wahrscheinlichkeitstheorie und verwandte Gebiete 61:467–481MathSciNetCrossRefzbMATHGoogle Scholar
  11. Devroye L, Krzyżak A (1989) An equivalence theorem for \({L}_1\) convergence of the kernel regression estimate. J Stat Plan Inference 23:71–82CrossRefzbMATHGoogle Scholar
  12. Devroye L, Wagner TJ (1980) Distribution-free consistency results in nonparametric discrimination and regression function estimation. Ann Stat 8:231–239MathSciNetCrossRefzbMATHGoogle Scholar
  13. Devroye L, Györfi L, Krzyżak A, Lugosi G (1994) On the strong universal consistency of nearest neighbor regression function estimates. Ann Stat 22:1371–1385MathSciNetCrossRefzbMATHGoogle Scholar
  14. Dvoretzky A, Kiefer J, Wolfowitz J (1956) Asymptotic minimax character of the sample distribution function and of the classical multinomial estimator. Ann Math Stat 27:642–669MathSciNetCrossRefzbMATHGoogle Scholar
  15. Egloff D, Leippold M (2010) Quantile estimation with adaptive importance sampling. Ann Stat 38:1244–1278MathSciNetCrossRefzbMATHGoogle Scholar
  16. Enss GC, Kohler M, Krzyżak A, Platz R (2016) Nonparametric quantile estimation based on surrogate models. IEEE Trans Inf Theory 62:5727–5739MathSciNetCrossRefzbMATHGoogle Scholar
  17. Falk M (1985) Asymptotic normality of the kernel quantile estimator. Ann Stat 13:428–433MathSciNetCrossRefzbMATHGoogle Scholar
  18. Glasserman P (2004) Monte Carlo methods in financial engineering. Springer, New YorkzbMATHGoogle Scholar
  19. Greblicki W, Pawlak M (1985) Fourier and Hermite series estimates of regression functions. Ann Inst Stat Math 37:443–454MathSciNetCrossRefzbMATHGoogle Scholar
  20. Györfi L (1981) Recent results on nonparametric regression estimate and multiple classification. Probl Control Inf Theory 10:43–52MathSciNetGoogle Scholar
  21. Györfi L, Kohler M, Krzyżak A, Walk H (2002) A distribution-free theory of nonparametric regression. Springer series in statistics. Springer, New YorkCrossRefzbMATHGoogle Scholar
  22. Hurtado J (2004) Structural reliability: statistical learning perspectives, vol 17. Lecture notes in applied and computational mechanics. Springer, BerlinCrossRefzbMATHGoogle Scholar
  23. Kaymaz I (2005) Application of Kriging method to structural reliability problems. Strut Saf 27:133–151CrossRefGoogle Scholar
  24. Kim S-H, Na S-W (1997) Response surface method using vector projected sampling points. Strut Saf 19:3–19CrossRefGoogle Scholar
  25. Kohler M (2000) Inequalities for uniform deviations of averages from expectations with applications to nonparametric regression. J Stat Plan Inference 89:1–23MathSciNetCrossRefzbMATHGoogle Scholar
  26. Kohler M, Krzyżak A (2001) Nonparametric regression estimation using penalized least squares. IEEE Trans Inf Theory 47:3054–3058MathSciNetCrossRefzbMATHGoogle Scholar
  27. Kohler M, Krzyżak A, Tent R, Walk H (2018) Nonparametric quantile estimation using importance sampling. Ann Inst Stat Math 70:439–465MathSciNetCrossRefzbMATHGoogle Scholar
  28. Lugosi G, Zeger K (1995) Nonparametric estimation via empirical risk minimization. IEEE Trans Inf Theory 41:677–687MathSciNetCrossRefzbMATHGoogle Scholar
  29. Massart P (1990) The tight constant in the Dvoretzky–Kiefer–Wolfowitz inequality. Ann Probab 18(3):1269–1283MathSciNetCrossRefzbMATHGoogle Scholar
  30. Morio J (2012) Extreme quantile estimation with nonparametric adaptive importance sampling. Simul Model Pract Theory 27:76–89CrossRefGoogle Scholar
  31. Nadaraya EA (1964) On estimating regression. Theory Probab Appl 9:141–142CrossRefzbMATHGoogle Scholar
  32. Nadaraya EA (1970) Remarks on nonparametric estimates for density functions and regression curves. Theory Probab Appl 15:134–137CrossRefzbMATHGoogle Scholar
  33. Neddermeyer JC (2009) Computationally efficient nonparametric importance sampling. J Am Stat Assoc 104(486):788–802MathSciNetCrossRefzbMATHGoogle Scholar
  34. Papadrakakis M, Lagaros N (2002) Reliability-based structural optimization using neural networks and Monte Carlo simulation. Comput Methods Appl Mech Eng 191:3491–3507CrossRefzbMATHGoogle Scholar
  35. Pepelyshev A, Rafajłowicz E, Steland A (2014) Estimation of the quantile function using Bernstein–Durrmeyer polynomials. J Nonparametr Stat 26:1–20MathSciNetCrossRefzbMATHGoogle Scholar
  36. Platz R, Enss GC (2015) Comparison of uncertainty in passive and active vibration isolation. In: Atamturktur H, Moaveni B, Papadimitriou C, Schoenherr T (eds) Model validation and uncertainty quantification, vol 3. Conference proceedings of the society for experimental mechanics series. Springer, BerlinGoogle Scholar
  37. Rafajłowicz E (1987) Nonparametric orthogonal series estimators of regression: a class attaining the optimal convergence rate in L2. Stat Probab Lett 5:219–224CrossRefzbMATHGoogle Scholar
  38. Sheather S, Marron S (1990) Kernel quantile estimators. J Am Stat Assoc 85:410–416MathSciNetCrossRefzbMATHGoogle Scholar
  39. Stone CJ (1977) Consistent nonparametric regression. Ann Stat 5:595–645MathSciNetCrossRefzbMATHGoogle Scholar
  40. Stone CJ (1982) Optimal global rates of convergence for nonparametric regression. Ann Stat 10:1040–1053MathSciNetCrossRefzbMATHGoogle Scholar
  41. Wahba G (1990) Spline models for observational data. SIAM, PhiladelphiaCrossRefzbMATHGoogle Scholar
  42. Watson GS (1964) Smooth regression analysis. Sankhya Ser A 26:359–372MathSciNetzbMATHGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Fachbereich MathematikTechnische Universität DarmstadtDarmstadtGermany

Personalised recommendations