Asymptotic properties of integrated square error and cross-validation for kernel estimation of a regression function

  • Peter Hall


We derive an asymptotic expansion of integrated square error in kernel-type nonparametric regression. A similar result is obtained for a cross-validatory estimate of integrated square error. Together these expansions show that cross-validation is asymptotically optimal in a certain sense.


Window Size Regression Function Nonparametric Regression Kernel Estimation Empiric Distribution Function 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Burkholder, D.L.: Distribution function inequalities for martingales. Ann. Probability 1, 19–42Google Scholar
  2. 2.
    Chow, Y.S., Geman, S., Wu, L.-D.: Consistent cross-validated density estimation. Ann. Statist 11, 25–38 (1983)MathSciNetCrossRefMATHGoogle Scholar
  3. 3.
    Collomb, G.: Estimation non-parametrique de la régression par la méthode du noyau. Thesis, Université Paul Sabatier, Toulouse (1976)Google Scholar
  4. 4.
    Collomb, G.: Quelques proprietes de la méthode du noyau pour l'estimation non-paramétrique de la régression en un point fixé. C. R. Acad. Sci. Paris 285A, 289–292 (1977)MathSciNetGoogle Scholar
  5. 5.
    Collomb, G.: Estimation non-paramétrique de la régression: revue bibliographique. Internat. Statist. Rev. 49, 75–93 (1981)MathSciNetCrossRefMATHGoogle Scholar
  6. 6.
    Craven, P., Wahba, G.: Smoothing noisy data with spline functions. Numer. Math. 31, 377–403 (1979)MathSciNetCrossRefMATHGoogle Scholar
  7. 7.
    Csörgő, M., Révész, P.: Strong Approximations in Probability and Statistics. New York and Budapest: Academic Press 1981Google Scholar
  8. 8.
    Devroye, L.P.: The uniform convergence of nearest neighbour regression function estimators and their application in optimization. IEEE Trans. Information Theory IT-24, 142–151 (1978)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Devroye, L.P.: The uniform convergence of the Nadaraya-Watson regression function estimate. Canad. J. Statist. 6, 179–191 (1979)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Devroye, L.P., Wagner, T.J.: Distribution free consistency results in nonparametric discrimination and regression function estimation. Ann. Statist. 8, 231–239 (1980)MathSciNetCrossRefMATHGoogle Scholar
  11. 11.
    Devroye, L.P., Wagner, T.J.: On the L 1 convergence of kernel estimators of regression functions with applications in discrimination. Z. Wahrscheinlichkeitstheorie verw. Geb. 51, 15–25 (1980)MathSciNetCrossRefMATHGoogle Scholar
  12. 12.
    Diaz, F.U.: Sur le choix du paramètre d'ajustement dans le lissage par functions spline. Numer. Math. 34, 15–28 (1980)MathSciNetCrossRefMATHGoogle Scholar
  13. 13.
    Golub, G.H., Heath, M., Wahba, G.: Generalized cross-validation as a method for choosing a good ridge parameter. Technometrics 21, 215–223 (1979)MathSciNetCrossRefMATHGoogle Scholar
  14. 14.
    Härdle, W., Marron, J.S.: Optimal bandwidth selection in nonparametric regression function estimation.Google Scholar
  15. 15.
    Komlós, J., Major, P., Tusnády, G.: An approximation of partial sums of independent random variables, and the sample distribution function. I. Z. Wahrscheinlichkeitstheorie verw. Geb. 32, 111–131 (1975)CrossRefMATHGoogle Scholar
  16. 16.
    Loève, M.: Probability Theory. Princeton, N.J.: Van Nostrand 1963MATHGoogle Scholar
  17. 17.
    Mack, Y.P., Silverman, B.W.: Weak and strong uniform consistency of kernel regression estimates. Z. Wahrscheinlichkeitstheorie verw. Geb. 61, 405–415 (1982)MathSciNetCrossRefMATHGoogle Scholar
  18. 18.
    Nadaraya, E.A.: On estimating regression. Theor. Probability Appl. 9, 141–142 (1964)CrossRefGoogle Scholar
  19. 19.
    Révész, P.: On the nonparametric estimation of the regression function. Problems of Control and Information Theory 8, 297–302Google Scholar
  20. 20.
    Schuster, E.F., Gregory, C.G.: On the nonconsistency of maximum likelihood nonparametric density estimators. In: Computer Science and Statistics: Proceedings of the 13th Symposium on the Interface. Ed. W.F. Eddy, 295–298. New York: Springer 1981CrossRefGoogle Scholar
  21. 21.
    Spiegelman, C., Sacks, J.: Consistent window estimation in nonparametric regression. Ann. Statist. 8, 240–246Google Scholar
  22. 22.
    Stone, C.J.: Consistent nonparametric regression. Ann. Statist. 5, 595–620 (1977)MathSciNetCrossRefMATHGoogle Scholar
  23. 23.
    Stone, C.J.: Optimal global rates of convergence for nonparametric regression. Ann. Statist. 10, 1040–1053 (1982)MathSciNetCrossRefMATHGoogle Scholar
  24. 24.
    Tusnády, G.: A remark on the approximation of the sample distribution function in the multidimensional case. Period. Math. Hungar. 8, 53–55 (1977)MathSciNetCrossRefMATHGoogle Scholar
  25. 25.
    Wahba, G., Wold, S.: A completely automatic French curve: fitting spline functions by crossvalidation. Comm. Statist. 4, 1–17 (1975)MathSciNetCrossRefMATHGoogle Scholar
  26. 26.
    Watson, G.S.: Smooth regression analysis. Sankhyā A26, 359–372 (1964)Google Scholar

Copyright information

© Springer-Verlag 1984

Authors and Affiliations

  • Peter Hall
    • 1
  1. 1.The Australian National UniversityCanberraAustralia

Personalised recommendations