Probability Theory and Related Fields

, Volume 92, Issue 1, pp 1–20

Smoothed cross-validation

  • Peter Hall
  • J. S. Marron
  • Byeong U. Park


For bandwidth selection of a kernel density estimator, a generalization of the widely studied least squares cross-validation method is considered. The essential idea is to do a particular type of “presmoothing” of the data. This is seen to be essentially the same as using the smoothed bootstrap estimate of the mean integrated squared error. Analysis reveals that a rather large amount of presmoothing yields excellent asymptotic performance. The rate of convergence to the optimum is known to be best possible under a wide range of smoothness conditions. The method is more appealing than other selectors with this property, because its motivation is not heavily dependent on precise asymptotic analysis, and because its form is simple and intuitive. Theory is also given for choice of the amount of presmoothing, and this is used to derive a data-based method for this choice.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Bickel, P., Ritov, Y.: Estimating integrated squared density derivatives. Sankhyā Ser. A.50, 381–393 (1988)Google Scholar
  2. Bowman, A. W.: An alternative method of cross-validation for the smoothing of density estimates. Biometrika71, 353–360 (1984)Google Scholar
  3. Burkholder, D.L.: Distribution function inequalities for martingales. Ann. Probab.1, 19–42 (1973)Google Scholar
  4. Burman, P.: A data dependent approach to density estimation, Z. Wahrscheinlichkeits theor. Verw. Geb.69, 609–628 (1985)Google Scholar
  5. Chiu, S.-T.: Bandwidth selection for kernel density estimation. Ann. Stat. to appear (1991)Google Scholar
  6. Devroye, L., Györfi, L.: Nonparametric density estimation: TheL 1 View. New York: Wiley 1984Google Scholar
  7. Diggle, P.J.: Statistical analysis of point patterns London: Academic Press 1983Google Scholar
  8. Diggle, P.J.: A kernel method for smoothing point process data. Appl. Stat.34, 138–147 (1985)Google Scholar
  9. Diggle, P.J., Marron, J.S.: Equivalence of smoothing parameter selectors in density and intensity estimation. J. Am. Stat. Assoc.83, 793–800 (1988)Google Scholar
  10. Fubank, R.L.: Spline smoothing and nonparametric regression. New York: Dekker 1988Google Scholar
  11. Faraway, J.J., Jhun, M.: Bootstrap choice of bandwidth for density estimation. J. Am. Stat. Assoc.85, 1119–1122 (1990)Google Scholar
  12. Gasser, T., Müller, H-G., Mammitzsch, V.: Kernels for nonparametric curve estimation. J. R. Stat. Soc., Ser.B47, 238–252 (1985)Google Scholar
  13. Härdle, W.: Applied nonparametric regression. Econometrics Society Monograph Series, No. 19, Cambridge: Cambridge University Press 1989Google Scholar
  14. Härdle, W., Hall, P., Marron, J.S.: How far are automatically chosen regression smoothers from their optimum? (with discussion). J. Am. Stat. Assoc.83, 86–95 (1988)Google Scholar
  15. Hall, P.: Objective methods for the estimation of window size in the nonparametric estimation of a density (unpublished manuscript, 1980)Google Scholar
  16. Hall, P.: Large sample optimality of least squares cross-validation in density estimation. Ann. Stat.11, 1156–1174 (1983)Google Scholar
  17. Hall, P.: Central limit theorem for integrated squared error of multivariate density estimators. J. Multivariate Anal.14, 1–16 (1984)Google Scholar
  18. Hall, P., Marron, J.S.: Extent to which least-squares cross-validation minimises integrated square error in nonparametric density estimation. Probab. Th. Rel. Fields74, 567–581 (1987a)Google Scholar
  19. Hall, P., Marron, J.S.: On the amount of noise inherent in bandwidth selection for a kernel density estimator. Ann. Stat. 15, 163–181 (1987b)Google Scholar
  20. Hall, P., Marron, J.S.: Estimation of integrated squared density derivatives. Stat. Probab. Lett.6, 109–115 (1987c)Google Scholar
  21. Hall, P., Marron, J.S.: Lower bounds for bandwidth selection in density estimation (unpublished manuscript, 1989)Google Scholar
  22. Hall, P., Sheather, S., Jones, M.C., Marron, J.S.: On optimal data-based bandwidth selection in kernel density estimation. (unpublished manuscript, 1989)Google Scholar
  23. Marron, J.S.: Automatic smoothing parameter selection: A survey. Emp. Econ.13, 187–208 (1988)Google Scholar
  24. Müller, H.G.: Empirical bandwidth choice for nonparametric kernel regression by means of pilot estimators. Stat. Decis.2, [Suppl] 193–206 (1985)Google Scholar
  25. Müller, H.G.: Nonparametric analysis of longitudinal data. Berlin Heidelberg New York: Springer 1988Google Scholar
  26. Park, B.U., Marron, J.S.: Comparison of data-driven bandwidth selectors. J. Am. Stat. Assoc.85, 66–72 (1990)Google Scholar
  27. Ripley, B.D.: Spatial statistics New York: Wiley 1981Google Scholar
  28. Rudemo, M.: Empirical choice of histograms and kernel density estimators. Scand. J. Stat.9, 65–78 (1982)Google Scholar
  29. Scott, D.W.: Averaged shifted histograms: effective nonparametric density estimation in several dimensions. Ann. Stat.4, 1024–1040 (1985)Google Scholar
  30. Scott, D.W., Factor, L.E.: Monte Carlo study of three data-based nonparametric density estimators. J. Am. Stat. Assoc.76, 9–15 (1981)Google Scholar
  31. Scott, D.W., Terrell, G.R.: Biased and unbiased cross-validation in density estimation. J. Am. Stat. Assoc.82, 1131–1146 (1987)Google Scholar
  32. Scott, D.W., Tapia, R.A., Thompson, J.W.: Kernel density estimation revisited. J. Nonlinear Anal., Theor. Methods Appl.1, 339–372 (1977)Google Scholar
  33. Sheather, S.J.: An improved data-based algorithm for choosing the window width when estimating the density at a point. Comput. Stat. Data Anal.4, 61–65 (1986)Google Scholar
  34. Silverman, B.W.: Density estimation for statistics and data analysis. New York: Chapman and Hall 1986Google Scholar
  35. Staniswallis, J.G.: Local bandwidth selection for kernel estimates. J. Am. Stat. Assoc.84, 284–288 (1987)Google Scholar
  36. Stone, C.J.: An asymptotically optimal window selection rule for kernel density estimates. Ann. Stat.12, 1285–1297 (1984)Google Scholar
  37. Taylor, C.C.: Bootstrap choice of the smoothing parameter in kernel density estimation Biometrika76, 705–712 (1989)Google Scholar
  38. Woodroofe, M.: On choosing a delta sequence. Ann. Math. Stat.41, 1665–1671 (1970)Google Scholar

Copyright information

© Springer-Verlag 1992

Authors and Affiliations

  • Peter Hall
    • 1
    • 2
  • J. S. Marron
    • 1
    • 2
  • Byeong U. Park
    • 1
    • 2
  1. 1.Australian National UniversityCanberraAustralia
  2. 2.Seoul National UniversityKorea

Personalised recommendations