Abstract
Regression function estimation from independent and identically distributed bounded data is considered. TheL 2 error with integration with respect to the design measure is used as an error criterion. It is shown that the kernel regression estimate with an arbitrary random bandwidth is weakly and strongly consistent forall distributions whenever the random bandwidth is chosen from some deterministic interval whose upper and lower bounds satisfy the usual conditions used to prove consistency of the kernel estimate for deterministic bandwidths. Choosing discrete bandwidths by cross-validation allows to weaken the conditions on the bandwidths.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Allen, D. M. (1974). The relationship between variable selection and data augmentation and a method for prediction,Technometrics,16, 125–127.
Devroye, L. (1981). On the almost everywhere convergence of nonparametric regression function estimates,Ann. Statist.,9, 1310–1319.
Devroye, L. and Györfi, L. (1985).Nonparametric Density Estimation: The L 1 View, Wiley, New York.
Devroye, L. and Krzyżak, A. (1989). An equivalence theorem forL 1 convergence of the kernel regression estimate,J. Statisti. Plann. Inference,23, 71–82.
Devroye, L. and Lugosi, G. (2001).Combinatorical Methods in Density Estimation. Springer, New York.
Devroye, L. P. and Wagner, T. J. (1980). Distribution-free consistency results in nonparametric discrimination and regression function estimation,Ann. Statist.,8, 231–239.
Devroye, L., Györfi, L., Krzyżak, A. and Lugosi, G. (1994). On the strong universal consistency of nearest neighbor regression function estimates,Ann. Statist.,22, 1371–1385.
Devroye, L., Györfi, L. and Lugosi, G. (1996).A Probabilistic Theory of Pattern Recognition, Springer, New York.
Dudley, R. (1978). Central limit theorems for empirical measures,Ann. Probab.,6, 899–929.
Efron, B. and Stein, C. (1980). The jackknife estimate of variance,Ann. Statist.,9, 586–596.
Fan, J. and Gijbels, I. (1996).Local Polynomial Modelling and Its Applications. Chapman & Hall, London.
Greblicki, W., Krzyżak, A. and Pawlak, M. (1984). Distribution-free pointwise consistency of kernel regression estimate,Ann. Statist.,12, 1570–1575.
Györfi, L. and Walk, H. (1996). On the strong universal consistency of a series type regression estimate,Math. Methods Statist. 5, 332–342.
Györfi, L. and Walk, H. (1997). On the strong universal consistency of a recursive regression estimate by Pál Révész.Statist. Probab. Lett.,31, 177–183.
Györfi, L., Kohler, M. and Walk, H. (1998). Weak and strong universal consistency of semi-recursive partitioning and kernel regression estimates,Statist. Decisions,16, 1–18.
Györfi, L., Kohler, M., Krzyżak, A. and Walk, H. (2002).A Distribution-free Theory of Nonparametric Regression, Springer Ser. Statist., Springer, New York.
Hamers, M. and Kohler, M. (2003). A bound on the expected maximal deviations of sample averages from their means,Statist. Probab. Lett.,62, 137–144.
Härdle, H. (1990).Applied Nonparametric Regression, Cambridge University Press, Cambridge.
Haussler, D. (1992). Decision theoretic generalizations of the PAC model for neural net and other learning applications,Inform. and Comput.,100, 78–150.
Hoeffding, W. (1963). Probability inequalities for sums of bounded random variables,J. Amer. Statist. Assoc.,58, 13–30.
Kohler, M. (1999). Universally consistent regression function estimation using hierarchical B-splines,J. Multivariate Anal.,67, 138–164.
Kohler, M. (2002). Universal consistency of local polynomial kernel regression estimates,Ann. Inst. Statist. Math.,54, 879–899.
Kohler, M. and Krzyżak, A. (2001). Nonparametric regression estimation using penalized least squares,IEEE Trans. Inform. Theory,47, 3054–3058.
Li, K. C. (1984). Consistency for cross-validated nearest neighbor estimates in nonparametric regression,Ann. Statist.,12, 230–240.
Lunts, A. and Brailovsky, V. (1967). Evaluation of attributes obtained in statistical decision rules,Engineering Cybernetics,3, 98–103.
McDiarmid, C. (1989). On the method of bounded differences,Surveys in Combinatorics, 148–188, Cambridge University Press, Cambridge.
Nobel, A. (1996). Histogram regression estimation using data-dependent partitions,Ann. Statist.,24, 1084–1105.
Simonoff, J. S. (1996).Smoothing Methods in Statistics, Springer, New York.
Spiegelman, C. and Sacks, J. (1980). Consistent window estimation in nonparametric regression,Ann. Statist.,8, 240–246.
Steele, J. (1986). An Efron-Stein inequality for nonsymmetric statistics,Ann. Statist.,14, 753–758.
Stone, C. J. (1977). Consistent nonparametric regression,Ann. Statist.,5, 595–645.
Stone, C. J. (1982). Optimal global rates of convergence for nonparametric regression.Ann. Statist.,10, 1040–1053.
Stone, M. (1974). Cross-validatory choice and assessment of statistical predictions (with discussion).J. Roy. Statist. Soc. Ser. B,36, 111–147.
Walk, H. (2002a). On cross-validation in kernel and partitioning regression estimation,Statist. Probab. Lett.,59, 113–123.
Walk, H. (2002b). Almost sure convergence properties of Nadaraya-Watson regression estimates,Modeling Uncertainty: An Examination of Its Theory, Methods and Applications (eds. M. Dror, P. L’Ecuyer and F. Szidarovszky), 201–223, Kluwer, Dordrecht.
Wong, W. H. (1983). On the consistency of cross-validation in kernel nonparametric regression,Ann. Statist.,11, 1136–1141.
Author information
Authors and Affiliations
Additional information
Research supported by DAAD, NSERC and Alexander von Humboldt Foundation.
The research of the second author was completed during his stay at the Technical University of Szczecin, Poland.
About this article
Cite this article
Kohler, M., Krzyżak, A. & Walk, H. Strong consistency of automatic kernel regression estimates. Ann Inst Stat Math 55, 287–308 (2003). https://doi.org/10.1007/BF02530500
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF02530500