Skip to main content
Log in

The stochastic approximation method for estimation of a distribution function

  • Published:
Mathematical Methods of Statistics Aims and scope Submit manuscript

Abstract

We apply the stochastic approximation method to construct a large class of recursive kernel estimators of a distribution function. We study the properties of these estimators and compare them with Nadaraya’s distribution estimator. It turns out that, with an adequate choice of the stepsize of the proposed algorithm, the MWISE (Mean Weighted Integrated Squared Error) of the proposed estimator is smaller than that of Nadaraya’s estimator. We corroborate these theoretical results by simulations and a real dataset.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  1. N. Altman and C. Leger, “Bandwidth Selection for Kernel Distribution Function Estimation”, J. Statist. Plann. Inference 46, 195–214 (1995).

    Article  MathSciNet  MATH  Google Scholar 

  2. A. Benveniste, M. Métivier, and P. Priouret, Adaptive Algorithm and Stochastic Approximations (Springer, Berlin, 1990).

    Book  Google Scholar 

  3. R. Bojanic and E. Seneta, “AUnified Theory of Regularly Varying Sequences”, Math. Z. 134, 91–106 (1973).

    Article  MathSciNet  MATH  Google Scholar 

  4. B. Delyon, “General Results on the Convergence of Stochastic Algorithms”, IEEE Trans. Automat. Control 41, 1245–1255 (1996).

    Article  MathSciNet  MATH  Google Scholar 

  5. M. Duflo, Random Iterative Models, in Applications of Mathematics (Springer, Berlin, 1997).

    Google Scholar 

  6. M. Falk, “Relative Efficiency and Deficiency of Kernel Type Estimator of Smooth Distribution Functions”, Statist. Neerl. 37, 73–83 (1983).

    Article  MathSciNet  MATH  Google Scholar 

  7. J. Galambos and E. Seneta, “Regularly Varying Sequences”, Amer. Math. Soc. 41, 110–116 (1973).

    Article  MathSciNet  MATH  Google Scholar 

  8. G. K. Golubev and B. Y. Levit, “On the Second Order Minimax Estimation of Distribution Functions”, Math. Meth. Statist. 5, 1–31 (1996).

    MathSciNet  MATH  Google Scholar 

  9. P. Hall and J. S. Maron, “Estimation of Integrated Squared Density Derivatives”, Statist. Probab. Lett. 6, 109–115 (1987).

    Article  MathSciNet  MATH  Google Scholar 

  10. P. D. Hill, “Kernel Estimation of a Distribution Function”, Commun. Statist. — Theor. Meth. 14, 605–620 (1985).

    Article  Google Scholar 

  11. E. Isogai and K. Hirose, “Nonparametric Recursive Kernel Estimators of a Distribution Function”, Bull. Inform. Cybernet. 26, 87–99 (1994).

    MathSciNet  MATH  Google Scholar 

  12. M. C. Jones, “The Performance of Kernel Density Functions in Kernel Distribution Function Estimation”, Statist. Probab. Lett. 9, 129–132 (1990).

    Article  MathSciNet  MATH  Google Scholar 

  13. J. Kiefer and J. Wolfowitz, “Stochastic Estimation of the Maximum of a Regression Function”, Ann. Math. Statist. 23, 462–466 (1952).

    Article  MathSciNet  MATH  Google Scholar 

  14. H. J. Kushner, “General Convergence Results for Stochastic Approximation via Week Convergence Theory”, J. Math. Anal. Appl. 61, 490–503 (1977).

    Article  MathSciNet  MATH  Google Scholar 

  15. H. J. Kushner and G. G. Yin, Stochastic Approximation and Recursive Algorithms and Applications, in Applications of Mathematics (Springer, New York, 2003), Vol. 35.

    Google Scholar 

  16. L. Ljung, “Strong Convergence of a Stochastic Approximation Algorithm”, Ann. Statist. 6, 680–696 (1978).

    Article  MathSciNet  MATH  Google Scholar 

  17. A. Mokkadem and M. Pelletier, “A Companion for the Kiefer-Wolfowitz-Blum Stochastic Approximation Algorithm”, Ann. Statist. 35, 1749–1772 (2007).

    Article  MathSciNet  MATH  Google Scholar 

  18. A. Mokkadem, M. Pelletier, and Y. Slaoui, “The Stochastic Approximation Method for the Estimation of a Multivariate Probability Density”, J. Statist. Plann. Inference. 139, 2459–2478 (2009).

    Article  MathSciNet  MATH  Google Scholar 

  19. A. Mokkadem, M. Pelletier, and Y. Slaoui, “Revisiting Révész’s Stochastic Approximation Method for the Estimation of a Regression Function”, ALEA Lat. Amer. J. Probab. Math. Statist. 6, 63–114 (2009).

    MathSciNet  MATH  Google Scholar 

  20. E. A. Nadaraya, “Some New Estimates for Distribution Functions”, Theory Probab. Appl. 9, 497–500 (1964).

    Article  Google Scholar 

  21. A. Quintela-del-Río and G. Estévez-Pérez, “Nonparametric Kernel Distribution Function Estimation with Kerdist: An R Package for Bandwidth Choice and Applications”, J. Statist. Softw. 50, 1–21 (2012).

    Google Scholar 

  22. R. D. Reiss, “Nonparametric Estimation of Smooth Distribution Function”, Scand. J. Statist. 8, 116–119 (1981).

    MathSciNet  MATH  Google Scholar 

  23. P. Révész, “Robbins-Monro Procedure in a Hilbert Space and Its Application in the Theory of Learning Processes I”, Studia Sci. Math. Hung. 8, 391–398 (1973).

    Google Scholar 

  24. P. Révész, “How to Apply the Method of Stochastic Approximation in the Nonparametric Estimation of a Regression Function”, Math. Operationsforsch. Statist., Ser. Statistics. 8, 119–126 (1977).

    MATH  Google Scholar 

  25. H. Robbins and S. Monro, “A Stochastic Approximation Method”, Ann. Statist. 22, 400–407 (1951).

    Article  MathSciNet  MATH  Google Scholar 

  26. D. Ruppert, “Almost Sure Approximations to the Robbins-Monro and Kiefer-Wolfowitz Processes with Dependent Noise”, Ann. Probab. 10, 178–187 (1982).

    Article  MathSciNet  MATH  Google Scholar 

  27. B.W. Silverman, Density Estimation for Statistics and Data Analysis (Chapman & Hall, London, 1986).

    Book  MATH  Google Scholar 

  28. Y. Slaoui, “Large and Moderate Principles for Recursive Kernel Density Estimators Defined by Stochastic Approximation Method”, Serdica Math. J. 39, 53–82 (2013).

    MathSciNet  Google Scholar 

  29. Y. Slaoui, “Bandwidth Selection for Recursive Kernel Density Estimators Defined by Stochastic Approximation Method”, J. Probab. Statist. 2014, ID 739640, doi:10.1155/2014/739640 (2014).

  30. A. B. Tsybakov, “Recurrent Estimation of the Mode of a Multidimensional Distribution”, Probl. Inf. Transm. 8, 119–126 (1990).

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Y. Slaoui.

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Slaoui, Y. The stochastic approximation method for estimation of a distribution function. Math. Meth. Stat. 23, 306–325 (2014). https://doi.org/10.3103/S1066530714040048

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.3103/S1066530714040048

Keywords

2000 Mathematics Subject Classification

Navigation