Skip to main content
Log in

Nonparametric adaptive estimation of conditional probabilities of rare events and extreme quantiles

  • Published:
Extremes Aims and scope Submit manuscript

Abstract

Let F t (x)=P(Xx|T=t) be the conditional distribution of a random variable X given that a covariate T takes the value \(t \in [0,T_{\max }],\) where we assume that the distributions F t are in the domain of attraction of the Fréchet distribution. We observe independent random variables \(X_{t_{1}},...,X_{t_{n}}\) associated to a sequence of times \(0\leq t_{1}<...<t_{n}\leq T_{\max },\) where \(X_{t_{i}}\) has the distribution function \(F_{t_{i}}.\) For each \(t\in [0,T_{\max }]\), we propose a nonparametric adaptive estimator for extreme tail probabilities and quantiles of F t . It follows from the Fisher-Tippett-Gnedenko theorem that the tail of the distribution function F t can be adjusted with a Pareto distribution of parameter 𝜃 t,τ starting from a threshold τ. We estimate the parameter 𝜃 t,τ using a nonparametric kernel estimator of bandwidth h based on the observations larger than τ and we propose a pointwise data driven procedure to choose the threshold τ. A global selection of the bandwidth h based on a cross-validation approach is given. Under some regularity assumptions, we prove that the non adaptive and adaptive estimators of 𝜃 t,τ are consistent and we determine their rate of convergence. Finally, we study this procedure using simulations and we analyze an environmental data set.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  • Allee, W.C., Emerson, A.E., Park, O., Park, T., Schmidt, K.P.: Principles of animal ecology. Springer, Philadelphia (1949). ISBN 978-0721611204

    Google Scholar 

  • Azaïs, R., Coudret, G., Durrieu, G.: A hidden renewal model for monitoring aquatic systems biosensors. Environmetrics 25, 189–199 (2014)

    Article  MathSciNet  Google Scholar 

  • Beirlant, J., Goegebeur, Y.: Regression with response distributions of pareto-type. Comput. Stat. Data Anal. 42, 595–619 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  • Beirlant, J., Goegebeur, Y.: Local polynomial maximum likelihood estimation for pareto-type distributions. J. Multivar. Anal. 89, 97–118 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  • Beirlant, J., Goegebeur, Y., Teugels, J., Segers, J.: Statistics of extremes: theory and applications. Wiley, Chichester (2004)

    Book  Google Scholar 

  • Burge, C.A., Judah, L.R., Conquest, L.L., Griffin, F.J., et al.: Summer seed mortality of the pacific oyster, crassostrea gigas thunberg grown in Tomales Bay, California, USA: the influence of oyster stock, planting time, pathogens, and environmental stressors. J. Shellfish Res. 26(1), 163–172 (2007)

    Article  Google Scholar 

  • Carreau, J., Girard, S.: Spatial extreme quantile estimation using a weighted log-likelihood approach. Journal de la Société Française de Statistique 152(1), 66–83 (2011)

    MathSciNet  Google Scholar 

  • Chambon, C., Legeay, A., Durrieu, G., Gonzalez, P., Ciret, P., Massabuau, J.-C.: Influence of the parasite worm Polydora sp. on the behaviour of the oyster crassostrea gigas: a study of the respiratory impact and associated oxidative stress. Mar. Biol. 152(2), 329–338 (2007)

    Article  Google Scholar 

  • Coudret, R., Durrieu, G., Saracco, J.: Comparison of kernel density estimators with assumption on number of modes. Commun. Stat. Simul. Comput. 44(1), 196–216 (2015)

    Article  MathSciNet  Google Scholar 

  • Davison, A.C., Ramesh, I.: N. Local likelihood smoothing of samples extremes. J. Roy. Statist. Soc. B 62(1), 191–208 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  • Davison, A.C., Smith, R.L.: Models for exceedances over high thresholds. J. Roy. Statist. Soc. B 52(3), 393–442 (1990)

    MathSciNet  MATH  Google Scholar 

  • de Haan, L., Ferreira, A.: Extreme value theory : an introduction. Springer series in operations research and financial engineering. Springer, New York (2006). ISBN 0-387-23946-4

  • Denny, M.W., Hunt, L.J.H., Miller, L.P., Harley, C.D.G.: On the prediction of extreme ecological events. Ecol. Monogr. 79(3), 397–421 (2009)

    Article  Google Scholar 

  • Doherty, F.G., Cherry, D.S., Cairns, J.: Valve closure responses of the asiatic clam c. fluminea exposed to cadmium and zinc. Hydrobiologia 153, 159–167 (1987)

    Article  Google Scholar 

  • Drees, H., Kaufmann, E.: Selecting the optimal sample fraction in univariate extreme value estimation. Stochastic Process. Appl. 75, 149–172 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  • El Methni, J., Gardes, L., Girard, S., Guillou, A.: Estimation of extreme quantiles from heavy and light tailed distributions, journal of statistical planning and inference. J. Stat. Plann. Inference 142(10), 2735–2747 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  • Embrechts, P., Klüppelberg, C., Mikosh, T.: Modelling extremal events: for insurance and finance. Springer (1997)

  • Fawcett, D., Walshaw, D.: Improved estimation for temporally clustered extremes. Environmetrics 18(2), 173–188 (2007)

    Article  MathSciNet  Google Scholar 

  • Folke, C., Carpenter, S., Walker, B., Scheffer, M., Elmqvist, T., Gunderson, L., Holling, C.S.: Regime shifts, resilience, and biodiversity in ecosystem management. Annu. Rev. Ecol. Syst. 35, 557–581 (2004)

    Article  Google Scholar 

  • Gardes, L., Girard, S.: A moving window approach for nonparametric estimation of the conditional tail index. J. Multivar. Anal. 99, 2368–2388 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  • Gardes, L., Girard, S.: Conditional extremes from heavy -tailed distributuions: an application to the estimation of extreme rainfall return levels. Extremes 13, 177–204 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  • Gardes, L., Stupfler, G.: Estimation of the conditional tail index using a smoothed local hill estimator. Extremes 17, 45–75 (2014)

    Article  MathSciNet  Google Scholar 

  • Goegebeur, Y., Guillou, A., Schorgen, A.: Nonparametric regression estimation of conditional tails: the random covariate case. Statistics (2013). to appear

  • Grama, I., Spokoiny, V.: Pareto approximation of the tail by local exponential modeling. Buletinul Academiei de Stiinte a Republicii Moldova. Matematica 53(1), 1–22 (2007)

    MathSciNet  Google Scholar 

  • Grama, I., Spokoiny, V.: Statistics of extremes by oracle estimation. Ann. Stat. 36(4), 1619–1648 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  • Grama, I., Tricot, J.M., Petiot, J.F.: Estimation of the survival probabilities by adjusting a cox model to the tail. C.R. Acad. Sci. Paris, Ser. I. 349(13-14), 807–811 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  • Grama, I., Tricot, J.M., Petiot, J.F.: Long term survival probabilities and Kaplan-Meier estimator. In: Proceedings of the 2013 Joint Statistical Meetings, pp 2066–2079 (2013)

  • Grama, I., Tricot, J.M., Petiot, J.F.: Estimation of the extreme survival probabilities from censored data. Buletinul Academiei de Stiinte a Republicii Moldova. Matematica 74(1), 33–62 (2014)

    MathSciNet  Google Scholar 

  • Guillou, A., Hall, P.: A diagnostic for selecting the threshold in extreme-value analysis. J. Roy. Statist. Soc. Ser. B 63, 293–305 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  • Hall, P.: On some simple estimates of an exponent of regular variation. J. Roy. Statist. Soc. B 44(1), 37–42 (1982)

    MATH  Google Scholar 

  • Hall, P., Tajvidi, N: Nonparametric analysis of temporal trend when fitting parametric models to extreme-value data. Stat. Sci. 15, 153–167 (2000)

    Article  MathSciNet  Google Scholar 

  • Hall, P., Welsh, A.H.: Best attainable rates of convergence for estimates of parameters of regular variation. Ann. Statist. 12(3), 1079–1084 (1984)

    Article  MathSciNet  MATH  Google Scholar 

  • Hall, P., Welsh, A.H.: Adaptive estimates of parameters of regular variation. Ann. Statist. 13(1), 331–341 (1985)

    Article  MathSciNet  MATH  Google Scholar 

  • Huisman, R., Koedijk, C.G., Kool, C.J.M., Palm, F.: Tail index estimates in small samples. J. Bus. Econ. Stat. 19(2), 208–216 (2001)

    Article  MathSciNet  Google Scholar 

  • Diebolt, J.J., Guillou, A., Ribereau, P.: Asymptotic normality of extreme quantile estimators based on the peaks-over-threshold approach. Comm. Statist. Theory and Methods 36(5), 869–886 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  • Jou, L.J., Liao, C.M.: A dynamic artificial clam (corbicula fluminea) allows parcimony on-line measurement of waterborne metals. Environ. Pollut. 144, 172–183 (2006)

    Article  Google Scholar 

  • Loader, C.: Local regression and likelihood. Springer (1999)

  • Nagai, K., Honjo, T., Go, J., Yamashita, H., Oh, S.J.: Detecting the shellfish killer heterocapsa circularisquama (dinophyceae) by measuring the bivalve valve activity with hall element sensor. Aquaculture 255, 395–401 (2006)

    Article  Google Scholar 

  • R Development Core Team: R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria (2012). ISBN 3-900051-07-0. http://www.R-project.org

    Google Scholar 

  • Robson, A., Wilson, R., Garcia de Leaniz, C.: Mussels flexing their muscles: a new method for quantifying bivalve behavior. Mar. Biol. 151, 1195–1204 (2007)

    Article  Google Scholar 

  • Samain, J.F., McCombie, H.: Summer mortality of the Pacific oyster Crassostrea gigas: the Morest Project. Quae, Versailles (2008). ISBN 978-2-7592-0054-2

    Google Scholar 

  • Schmitt, F.G., De Rosa, M., Durrieu, G., Sow, M., Ciret, P., Tran, D., Massabuau, J.C.: Statistical analysis of bivalve high frequency microclosing behavior: scaling properties and shot noise modeling. Int. J. Bifurcation Chaos 21 (12), 3565–3576 (2011)

    Article  Google Scholar 

  • Smith, R.L.: Extreme value analysis of environmental time series: An application to trend detection in ground-level ozone. Stat. Sci. 4(4), 367–393 (1989)

    Article  MATH  Google Scholar 

  • Sow, M., Durrieu, G., Briollais, L.: Water quality assessment by means of HFNI valvometry and high-frequency data modeling. Environmental Monitoring and Assessment 182(1-4), 155–170 (2011)

    Article  Google Scholar 

  • Spokoiny, V.: Multiscale local change point detection with applications to value-at-risk. Ann. Statist. 37(3), 1405–1436 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  • Staniswalis, J.G.: The kernel estimate of a regression function in likelihood-based models. J. Am. Stat. Assoc. 84(405), 276–283 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  • Stone, C.J.: Optimal global rates of convergence for nonparametric regression. Ann. Statist. 10(4), 1040–1053 (1982)

    Article  MathSciNet  MATH  Google Scholar 

  • Stupfler, G.: A moment estimator for the conditional extreme-value index. Electron. J. Stat. 7, 1935–2343 (2013)

    Article  MathSciNet  Google Scholar 

  • Tran, D., Ciret, P., Ciutat, A., Durrieu, G, Massabuau, J.-C.: Estimation of potential and limits of bivalve closure response to detect contaminants: application to cadmium. Environ. Toxicol. Chem. 22(4), 914–920 (2003)

    Article  Google Scholar 

  • Tran, D., Fournier, E., Durrieu, G., Massabuau, J.-C.: Copper detection in the Asiatic clam Corbicula fluminea: Optimum valve closure response. Aquat. Toxicol. 66, 333–343 (2004)

    Article  Google Scholar 

  • Tran, D., Fournier, E., Durrieu, G., Massabuau, J.-C.: Inorganic mercury detection by valve closure response in the freshwater clam corbicula fluminea: integration of time and water metal concentration changes. Environ. Toxicol. Chem. 26, 1545–1551 (2007)

    Article  Google Scholar 

  • Tran, D., Haberkorn, H., Soudant, P., Ciret, P., Massabuau, J.-C.: Behavioral responses of Crassostrea gigas exposed to the harmful algae Alexandrium minutum. Aquaculture 298, 338–345 (2010)

    Article  Google Scholar 

  • Weissman, I.: Estimation of parameters and large quantiles based on the k largest observations. J. Am. Stat. Assoc. 73(364), 812–815 (1978)

    MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Gilles Durrieu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Durrieu, G., Grama, I., Pham, QK. et al. Nonparametric adaptive estimation of conditional probabilities of rare events and extreme quantiles. Extremes 18, 437–478 (2015). https://doi.org/10.1007/s10687-015-0219-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10687-015-0219-z

Keywords

AMS 2000 Subject Classifications

Navigation