Skip to main content
SpringerLink
Log in

On dealing with the unknown population minimum in parametric inference

  • Original Paper
  • Published:
AStA Advances in Statistical Analysis Aims and scope Submit manuscript

Abstract

A myriad of physical, biological and other phenomena are better modeled with semi-infinite distribution families, in which case not knowing the population minimum becomes a hassle when performing parametric inference. Ad hoc methods to deal with this problem exist, but are suboptimal and sometimes unfeasible. Besides, having the statistician handcraft solutions in a case-by-case basis is counterproductive. In this paper, we propose a framework under which the issue can be analyzed, and perform an extensive search in the literature for methods that could be used to solve the aforementioned problem; we also propose a method of our own. Simulation experiments were then performed to compare some methods from the literature and our proposal. We found that the straightforward method, which is to infer the population minimum by maximum likelihood, has severe difficulty in giving a good estimate for the population minimum, but manages to achieve very good inferred models. The other methods, including our proposal, involve estimating the population minimum, and we found that our method is superior to the other methods of this kind, considering the distributions simulated, followed very closely by the endpoint estimator by Alves et al. (Stat Sin 24(4):1811–1835, 2014). Although these two give much more accurate estimates for the population minimum, the straightforward method also displays some advantages, so choosing between these three methods will depend on the problem domain.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  • Alexopoulos, C., Goldsman, D., Mokashi, A.C., Tien, K.W., Wilson, J.R.: Sequest: a sequential procedure for estimating quantiles in steady-state simulations. Oper. Res. 67(4), 1162–1183 (2019)

    MathSciNet  MATH  Google Scholar 

  • Alves, I.F., Neves, C.: Estimation of the finite right endpoint in the gumbel domain. Stat. Sin. 24(4), 1811–1835 (2014)

    MathSciNet  MATH  Google Scholar 

  • Alves, I.F., de Haan, L., Neves, C.: How far can man go? In: Torelli, N., Pesarin, F., Bar-Hen, A. (eds) Advances in Theoretical and Applied Statistics, pp. 187–197. Springer, Berlin (2013)

    Google Scholar 

  • Alves, I.F., Neves, C., Rosário, P.: A general estimator for the right endpoint with an application to supercentenarian women’s records. Extremes 20(1), 199–237 (2017)

  • Anderson, D., Burnham, K.: Model Selection and Multi-model Inference. Springer, Berlin (2004)

    Google Scholar 

  • Athreya, K.B., Fukuchi, J.i.: Confidence intervals for endpoints of a cdf via bootstrap. J. Stat. Plan. Inference 58(2), 299–320 (1997)

    Article  MATH  Google Scholar 

  • Beirlant, J., Goegebeur, Y., Segers, J., Teugels, J.L.: Statistics of Extremes: Theory and Applications. Wiley, New York (2004)

    Book  MATH  Google Scholar 

  • Beirlant, J., Bouquiaux, C., Werker, B.J.: Semiparametric lower bounds for tail index estimation. J. Stat. Plan. Inference 136(3), 705–729 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  • Beranger, B., Padoan, S.A., Sisson, S.A.: Estimation and uncertainty quantification for extreme quantile regions. Extremes 24(2), 349–375 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  • Cai, J.J., de Haan, L., Zhou, C.: Bias correction in extreme value statistics with index around zero. Extremes 16(2), 173–201 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  • Chung, J., Kannappan, P., Ng, C., Sahoo, P.: Measures of distance between probability distributions. J. Math. Anal. Appl. 138(1), 280–292 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  • Cordeiro, G.M., de Castro, M.: A new family of generalized distributions. J. Stat. Comput. Simul. 81(7), 883–898 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  • Daouia, A., Simar, L.: Nonparametric efficiency analysis: a multivariate conditional quantile approach. J. Econom. 140(2), 375–400 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  • Dekkers, A.L., Einmahl, J.H., De Haan, L.: A moment estimator for the index of an extreme-value distribution. Ann. Stat. 17(4), 1833–1855 (1989)

    MathSciNet  MATH  Google Scholar 

  • Diebolt, J., Gardes, L., Girard, S., Guillou, A.: Bias-reduced estimators of the weibull tail-coefficient. TEST 17(2), 311–331 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  • Dierckx, G., Beirlant, J., De Waal, D., Guillou, A.: A new estimation method for weibull-type tails based on the mean excess function. J. Stat. Plan. Inference 139(6), 1905–1920 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  • Drees, H.: On smooth statistical tail functionals. Scand. J. Stat. 25(1), 187–210 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  • Drees, H., et al.: Extreme quantile estimation for dependent data, with applications to finance. Bernoulli 9(4), 617–657 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  • Falk, M.: Some best parameter estimates for distributions with finite endpoint. Stat. J. Theor. Appl. Stat. 27(1–2), 115–125 (1995)

    MathSciNet  MATH  Google Scholar 

  • Ferreira, A., Haan, L.d., Peng, L.: On optimising the estimation of high quantiles of a probability distribution. Statistics 37(5), 401–434 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  • Gardes, L., Girard, S., Guillou, A.: Weibull tail-distributions revisited: a new look at some tail estimators. J. Stat. Plan. Inference 141(1), 429–444 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  • Gibbs, A.L., Su, F.E.: On choosing and bounding probability metrics. Int. Stat. Rev. 70(3), 419–435 (2002)

    Article  MATH  Google Scholar 

  • Girard, S., Guillou, A., Stupfler, G.: Estimating an endpoint with high-order moments. Test 21(4), 697–729 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  • Girard, S., Guillou, A., Stupfler, G.: Estimating an endpoint with high order moments in the weibull domain of attraction. Stat. Probab. Lett. 82(12), 2136–2144 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  • Goegebeur, Y., Beirlant, J., De Wet, T.: Generalized kernel estimators for the Weibull-tail coefficient. Commun. Stat. Theory Methods 39(20), 3695–3716 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  • Goldberg, D.E.: Fundamentals of Chemistry. McGraw-Hill, New York (2006)

    Google Scholar 

  • Goldenshluger, A., Tsybakov, A.: Estimating the endpoint of a distribution in the presence of additive observation errors. Stat. Probab. Lett. 68(1), 39–49 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  • Haan, L.D., Ferreira, A.: Extreme Value Theory: An Introduction. Springer, Berlin (2006)

    Book  MATH  Google Scholar 

  • Haan, L., Stadtmüller, U.: Generalized regular variation of second order. J. Aust. Math. Soc. 61(3), 381–395 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  • Hall, P.: On estimating the endpoint of a distribution. Ann. Stat. 10(2), 556–568 (1982)

    Article  MathSciNet  MATH  Google Scholar 

  • Hall, P., Park, B.U.: New methods for bias correction at endpoints and boundaries. Ann. Stat. 30(5), 1460–1479 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  • Hall, P., Wang, J.Z.: Estimating the End-Point of a Probability Distribution Using Minimum-Distance Methods, pp. 177–189. Bernoulli, Basel (1999)

    MATH  Google Scholar 

  • Hellinger, E.: Die orthogonalinvarianten quadratischer formen von unendlichvielen variabelen. Ph.D. thesis, University of Göttingen (1907)

  • Hill, B.M.: A simple general approach to inference about the tail of a distribution. Ann. Stat. 3(5), 1163–1174 (1975)

    Article  MathSciNet  MATH  Google Scholar 

  • Hosking, J.R., Wallis, J.R.: Parameter and quantile estimation for the generalized pareto distribution. Technometrics 29(3), 339–349 (1987)

    Article  MathSciNet  MATH  Google Scholar 

  • Jenkinson, A.F.: The frequency distribution of the annual maximum (or minimum) values of meteorological elements. Q. J. R. Meteorol. Soc. 81(348), 158–171 (1955)

    Article  Google Scholar 

  • Kahaner, D., Moler, C., Nash, S.: Numerical Methods and Software. Prentice-Hall, London (1989)

    MATH  Google Scholar 

  • Lawless, J.F.: Statistical Models and Methods for Lifetime Data. Wiley, New York (2003)

    MATH  Google Scholar 

  • Leng, X., Peng, L., Wang, X., Zhou, C.: Endpoint estimation for observations with normal measurement errors. Extremes 22(1), 71–96 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  • Li, D., Peng, L.: Does bias reduction with external estimator of second order parameter work for endpoint? J. Stat. Plan. Inference 139(6), 1937–1952 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  • Li, D., Peng, L.: Comparing extreme models when the sign of the extreme value index is known. Stat. Probab. Lett. 80(7–8), 739–746 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  • Li, Z., Peng, L.: Bootstrapping endpoint. Sankhya A 74(1), 126–140 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  • Li, D., Peng, L., Qi, Y.: Empirical likelihood confidence intervals for the endpoint of a distribution function. TEST 20(2), 353–366 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  • Li, D., Peng, L., Xu, X.: Bias reduction for endpoint estimation. Extremes 14(4), 393–412 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  • Lindsay, B.G.: Mixture models: theory, geometry and applications. In: NSF-CBMS Regional Conference Series in Probability and Statistics, pp. i–163. JSTOR (1995)

  • Loh, W.Y.: Estimating an endpoint of a distribution with resampling methods. Ann. Stat. 12(4), 1543–1550 (1984)

    Article  MathSciNet  MATH  Google Scholar 

  • Meister, A., Neumann, M.H.: Deconvolution from non-standard error densities under replicated measurements. Stat. Sin. 20(4), 1609–1636 (2010)

    MathSciNet  MATH  Google Scholar 

  • Methni, J.E., Gardes, L., Girard, S., Guillou, A.: Estimation of extreme quantiles from heavy and light tailed distributions. J. Stat. Plan. Inference 142(10), 2735–2747 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  • Mood, A.M.: Introduction to the Theory of Statistics. McGraw-Hill, New York (1950)

    MATH  Google Scholar 

  • Mudholkar, G.S., Srivastava, D.K.: Exponentiated Weibull family for analyzing bathtub failure-rate data. IEEE Trans. Reliab. 42(2), 299–302 (1993)

    Article  MATH  Google Scholar 

  • Müller, S., Hüsler, J.: Iterative estimation of the extreme value index. Methodol. Comput. Appl. Probab. 7(2), 139–148 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  • Pickands, J.: Statistical inference using extreme order statistics. Ann. Stat. 3(1), 119–131 (1975)

    MathSciNet  MATH  Google Scholar 

  • Royden, H.L.: Real Analysis. Prentice Hall, London (1988)

    MATH  Google Scholar 

  • Smith, R.L.: Maximum likelihood estimation in a class of nonregular cases. Biometrika 72(1), 67–90 (1985)

    Article  MathSciNet  MATH  Google Scholar 

  • Smith, R.L.: Estimating tails of probability distributions. Ann. Stat. 15(3), 1174–1207 (1987)

    Article  MathSciNet  MATH  Google Scholar 

  • Stacy, E.W., Mihram, G.A.: Parameter estimation for a generalized gamma distribution. Technometrics 7(3), 349–358 (1965)

    Article  MathSciNet  MATH  Google Scholar 

  • Stefanski, L.A., Carroll, R.J.: Deconvolving kernel density estimators. Statistics 21(2), 169–184 (1990)

    Article  MathSciNet  MATH  Google Scholar 

  • Torabi, H., Montazeri, N.H.: The logistic-uniform distribution and its applications. Commun. Stat. Simul. Comput. 43(10), 2551–2569 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  • Vaart, A.W.V.D.: Asymptotic Statistics, vol. 3. Cambridge University Press, Cambridge (2000)

    Google Scholar 

  • Valk, C.d.: Approximation and estimation of very small probabilities of multivariate extreme events. Extremes 19(4), 687–717 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  • Valk, C.d.: Approximation of high quantiles from intermediate quantiles. Extremes 19(4), 661–686 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  • Valk, C.D., Cai, J.: A high quantile estimator based on the log-generalized Weibull tail limit. Econom. Stat. 6, 107–128 (2018)

    MathSciNet  Google Scholar 

  • Vapnik, V.N.: Statistical Learning theory. Wiley, Berlin (1998)

    MATH  Google Scholar 

  • Wang, F., Peng, L., Qi, Y., Xu, M.: Maximum penalized likelihood estimation for the endpoint and exponent of a distribution. Stat. Sin. 29(1), 203–224 (2019)

    MathSciNet  MATH  Google Scholar 

  • Yazidi, A., Hammer, H.: Multiplicative update methods for incremental quantile estimation. IEEE Trans. Cybern. 49(3), 746–756 (2017)

    Article  Google Scholar 

Download references

Acknowledgements

We are grateful to prof. Ricardo Marcacini for valuable suggestions, as well as the LaSDPC and BioCom laboratories for the computational and other resources. We also thank the Coordination for the Improvement of Higher Education Personnel (CAPES) for the financial support and also CeMEAI (FAPESP Grant 2013/07375-0) for providing access to their supercomputer.

Funding

This project is funded by the Coordination for the Improvement of Higher Education Personnel (CAPES).

Author information

Authors and Affiliations

  1. Institute of Mathematics and Computer Sciences, University of São Paulo, São Paulo, Brazil

    Matheus Henrique Junqueira Saldanha & Adriano Kamimura Suzuki

Authors
  1. Matheus Henrique Junqueira Saldanha
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Adriano Kamimura Suzuki
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

MHJ Saldanha and AK Suzuki were involved in conceptualization, methodology and formal analysis and investigation; MHJ Saldanha wrote and prepared the original draft, prepared the materials and collected the data; and AK Suzuki was responsible for writing, reviewing and editing, and supervision.

Corresponding author

Correspondence to Matheus Henrique Junqueira Saldanha.

Ethics declarations

Conflict of interest

The authors declare there is no conflict of interest.

Code availability

The appropriate computer programs were submitted with this paper for review.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Saldanha, M.H.J., Suzuki, A.K. On dealing with the unknown population minimum in parametric inference. AStA Adv Stat Anal 107, 509–535 (2023). https://doi.org/10.1007/s10182-022-00445-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10182-022-00445-9

Keywords

Mathematics Subject Classification

Search

Navigation