Abstract
Regression function estimation from independent and identically distributed data is considered. The L 2 error with integration with respect to the design measure is used as an error criterion. It is shown that suitably defined local polynomial kernel estimates are weakly and strongly universally consistent, i.e., it is shown that the L 2 errors of these estimates converge to zero almost surely and in L 1 for all distributions.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Devroye, L. and Krzyżak, A. (1989). An equivalence theorem for L 1 convergence of the kernel regression estimate, J. Statist. Plann. Inference, 23, 71–82.
Devroye, L. P. and Wagner, T. J. (1980). Distribution-free consistency results in nonparametric discrimination and regression function estimation, Ann. Statist., 8, 231–239.
Devroye, L., Györfi, L., Krzyżak, A. and Lugosi, G. (1994). On the strong universal consistency of nearest neighbor regression function estimates, Ann. Statist., 22, 1371–1385.
Devroye, L., Györfi, L. and Lugosi, G. (1996). A Probabilistic Theory of Pattern Recognition, Springer, New York.
Dudley, R. (1978). Central limit theorems for empirical measures, Ann. Probab., 6, 899–929.
Eubank, R. L. (1988). Spline Smoothing and Nonparametric Regression, Marcel Dekker, New York.
Fan, J. and Gijbels, I. (1996). Local Polynomial Modelling and Its Applications, Chapman & Hall, London.
Györfi, L. (1991). Universal consistency of a regression estimate for unbounded regression functions, Nonparametric Functional Estimation and Related Topics (ed. G. Roussas), NATO ASI Series, 329–338, Kluwer, Dordrecht.
Györfi, L. and Walk, H. (1996). On the strong universal consistency of a series type regression estimate, Math. Methods Statist., 5, 332–342.
Györfi, L. and Walk, H. (1997). On the strong universal consistency of a recursive regression estimate by Pál Révész, Statist. Probab. Lett., 31, 177–183.
Györfi, L., Kohler, M. and Walk, H. (1998). Weak and strong universal consistency of semi-recursive partitioning and kernel regression estimates, Statist. Decisions, 16, 1–18.
Härdle, H. (1990). Applied Nonparametric Regression, Cambridge University Press, Cambridge, Massachusetts.
Haussler, D. (1992). Decision theoretic generalizations of the PAC model for neural net and other learning applications, Inform. and Comput., 100, 78–150.
Kohler, M. (1997). On the universal consistency of a least squares spline regression estimator, Math. Methods Statistics, 6, 349–364.
Kohler, M. (1999). Universally consistent regression function estimation using hierarchical B-splines, J. Multivariate Anal., 67, 138–164.
Kohler, M. and Krzyżak, A. (2001). Nonparametric regression estimation using penalized least squares, in IEEE Transactions on Information Theory, 47, 3054–3058.
Korostelev, A. P. and Tsybakov, A. B. (1993). Minimax Theory of Image ReConstruction, Springer, Berlin.
Krzyżak, A., Linder, T. and Lugosi, G. (1996). Nonparametric estimation and classification using radial basis function nets and empirical risk minimization, IEEE Transactions on Neural Networks, 7, 475–487.
Lugosi, G. and Zeger, K. (1995). Nonparametric estimation via empirical risk minimization, IEEE Trans. Inform. Theory, 41, 677–687.
Nobel, A. (1996). Histogram regression estimation using data-dependent partitions, Ann. Statist., 24, 1084–1105.
Spiegelman, C. and Sacks, J. (1980). Consistent window estimation in nonparametric regression, Ann. Statist., 8, 240–246.
Stone, C. J. (1977). Consistent nonparametric regression, Ann. Statist., 5, 595–645.
Stone, C. J. (1982). Optimal global rates of convergence for nonparametric regression, Ann. Statist., 10, 1040–1053.
Wahba, G. (1990). Spline Models for Observational Data, SIAM, Philadelphia, Pennsylvania.
Walk, H. (2002). Almost sure convergence properties of Nadaraya-Watson regression estimates, Essays on Uncertainty-S. Yakowitz Memorial Volume (eds. M. Dror, P. L'Ecuyer and F. Szidarovszky), 201–223, Kluwer, Dordrecht.
Author information
Authors and Affiliations
About this article
Cite this article
Kohler, M. Universal Consistency of Local Polynomial Kernel Regression Estimates. Annals of the Institute of Statistical Mathematics 54, 879–899 (2002). https://doi.org/10.1023/A:1022427805425
Issue Date:
DOI: https://doi.org/10.1023/A:1022427805425