Extent to which least-squares cross-validation minimises integrated square error in nonparametric density estimation

Probability Theory and Related Fields volume 74, pages567–581(1987)Cite this article

Summary

Let h o, ĥ o and ĥ c be the windows which minimise mean integrated square error, integrated square error and the least-squares cross-validatory criterion, respectively, for kernel density estimates. It is argued that ĥ o, not h o, should be the benchmark for comparing different data-driven approaches to the determination of window size. Asymptotic properties of h o-ĥ o and ĥ c -ĥ o, and of differences between integrated square errors evaluated at these windows, are derived. It is shown that in comparison to the benchmark ĥ o, the observable window ĥ c performs as well as the so-called “optimal” but unattainable window h o, to both first and second order.

This is a preview of subscription content, log in to check access.

References

  1. 1.

    Bickel, P.J., Rosenblatt, M.: On some global measures of the deviations of density estimates. Ann. Stat. 1, 1071–1095 (1973)

    Google Scholar 

  2. 2.

    Bowman, A.W.: A comparative study of some kernel-based nonparametric density estimates. Manchester-Sheffield School of Probability and Statistics Research Report 84/AWB/1 (1982)

  3. 3.

    Bowman, A.W.: An alternative method of cross-validation for the smoothing of density estimates. Biometrika 71, 353–360 (1984)

    Google Scholar 

  4. 4.

    Bowman, A.W., Hall, P., Titterington, D.M.: Cross-validation in nonparametric estimation of probabilities and probability densities. Biometrika 71, 341–352 (1984)

    Google Scholar 

  5. 5.

    Chow, Y.S., Geman, S., Wu, L.D.: Consistent cross-validated density estimation. Ann. Stat. 11, 25–38 (1983)

    Google Scholar 

  6. 6.

    Csörgö, M., Révész, P.: Strong approximations in probability and statistics. New York: Academic Press 1981

    Google Scholar 

  7. 7.

    Devroye, L., Györfi, L.: Nonparametric density estimation: the L 1 view. New York: Wiley 1985

    Google Scholar 

  8. 8.

    Duin, R.P.W.: On the choice of smoothing parameters for Parzen estimators of probability density functions. IEEE Trans. Comput. C25, 1175–1179 (1976)

    Google Scholar 

  9. 9.

    Habbema, J.D.F., Hermans, J., van den Broek, K.: A stepwise discriminant analysis program using density estimation. Compstat 1974, ed.: G. Bruckman 101–110. Vienna: Physica 1974

    Google Scholar 

  10. 10.

    Hall, P.: Limit theorems for stochastic measures of the accuracy of density estimators. Stochastic Processes Appl. 13, 11–25 (1982)

    Google Scholar 

  11. 11.

    Hall, P.: Large sample optimality of least squares cross-validation in density estimation. Ann. Stat. 11, 1156–1174 (1983)

    Google Scholar 

  12. 12.

    Hall, P.: Central limit theorem for integrated square error of multivariate nonparametric density estimators. J. Multivariate Anal. 14, 1–16 (1984)

    Google Scholar 

  13. 13.

    Hall, P.: Asymptotic theory of minimum integrated square error for multivariate density estimation. In: Krishnaiah, P.R. (ed.) Proc. Sixth Internat. Sympos. Multivariate Analysis, pp 289–309. Amsterdam: North Holland 1985

    Google Scholar 

  14. 14.

    Marron, J.S.: An asymptotically efficient solution to the bandwidth problem of kernel density estimation. Ann. Stat. 13, 1011–1023 (1985).

    Google Scholar 

  15. 15.

    Marron, J.S.: A comparison of cross-validation techniques in density estimation. N.C. Inst. of Statist. Mimeo Series #1568 (1985)

  16. 16.

    Rice, J.: Bandwidth choice for nonparametric regression. Ann. Stat. 12, 1215–1230 (1984)

    Google Scholar 

  17. 17.

    Rosenblatt, M.: Curve estimates. Ann. Math. Stat. 42, 1815–1842 (1971)

    Google Scholar 

  18. 18.

    Rosenblatt, M.: A quadratic measure of deviation of two-dimensional density estimates and a test of independence. Ann. Stat. 3, 1–14 (1975)

    Google Scholar 

  19. 19.

    Rudemo, M.: Empirical choice of histogram and kernel density estimators. Scand. J. Stat. 9, 65–78 (1982)

    Google Scholar 

  20. 20.

    Silverman, B.W.: Weak and strong uniform consistency of the kernel estimate of a density and its derivatives. Ann. Stat. 6, 177–184 (1978)

    Google Scholar 

  21. 21.

    Stone, C.J.: An asymptotically optimal window selection rule for kernel density estimates. Ann. Stat. 12, 1285–1297 (1984)

    Google Scholar 

  22. 22.

    Titterington, D.M.: Common structure of smoothing techniques in statistics. Int. Stat. Rev. 53, 141–170 (1985)

    Google Scholar 

  23. 23.

    Woodroofe, M.: On choosing a delta sequence. Ann. Math. Stat. 41, 1665–1671 (1970)

    Google Scholar 

Download references

Author information

Affiliations

  1. Dept. of Statistics, Faculty of Economics and Commerce, Australian National University, GPO Box 4, 2601, Canberra, ACT, Australia

    Peter Hall

  2. University of North Carolina, Chapel Hill

    James Stephen Marron

Authors
  1. Peter Hall
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. James Stephen Marron
    View author publications

    You can also search for this author in PubMed Google Scholar

Additional information

On leave from Australian National University. — Work of first author supported by U.S.A.F. Grant No. F 49620 82 C 009.

Research of second author partially supported by NSF Grant DMS-8400602.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Hall, P., Marron, J.S. Extent to which least-squares cross-validation minimises integrated square error in nonparametric density estimation. Probab. Th. Rel. Fields 74, 567–581 (1987). https://doi.org/10.1007/BF00363516

Download citation

Keywords

  • Stochastic Process
  • Probability Theory
  • Window Size
  • Statistical Theory
  • Density Estimate