Abstract
For semi-recursive and recursive kernel estimates of a regression of Y on X (d-dimensional random vector X, integrable real random variable Y), introduced by Devroye and Wagner and by Révész, respectively, strong universal pointwise consistency is shown, i.e. strong consistency P X -almost everywhere for general distribution of (X, Y). Similar results are shown for the corresponding partitioning estimates.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Ahmad, I. A. and Lin, P.-E. (1976). Nonparametric sequential estimation of a multiple regression function, Bulletin of Mathematical Statistics, 17, 63-75.
Algoet, P. and Györfi, L. (1999). Strong universal pointwise consistency of some regression function estimates, J. Multivariate Anal., 71, 125-144.
de Guzmán, M. (1970). A covering lemma with applications to differentiability of measures and singular integral operators, Studia Math., 34, 299-317.
Devroye, L. (1981). On the almost everywhere convergence of nonparametric regression function estimates, Ann. Statist., 9, 1310-1319.
Devroye, L. and Györfi, L. (1983). Distribution-free exponential bound on the L 1 error of partitioning estimates of a regression function, Proceedings of the Fourth Pannonian Symposium on Mathematical Statistics (eds. F. Konecny, J. Mogyoródi, and W. Wertz), 67-76, Akadémiai Kiadó, Budapest.
Devroye, L. and Krzyżak, A. (1989). An equivalence theorem for L 1 convergence of the kernel regression estimate, J. Statist. Plann. Inference, 23, 71-82.
Devroye, L. and Wagner, T. J. (1980a). Distribution-free consistency results in nonparametric discrimination and regression function estimation, Ann. Statist., 8, 231-239.
Devroye, L. and Wagner, T. J. (1980b). On the L 1 convergence of kernel estimators of regression functions with applications in discrimination, Z. Wahrsch. Verw. Gebiete, 51, 15-25.
Greblicki, W. (1974). Asymptotically optimal probabilistic algorithms for pattern recognition and identification, Prace Nauk. Inst. Cybernet. Tech. Politech. Wroclaw., No. 18., Ser. Monogr., No. 3.
Greblicki, W. and Pawlak, M. (1985). Fourier and Hermite series estimates of regession functions, Ann. Inst. Statist. Math., 37, 443-454.
Greblicki, W. and Pawlak, M. (1987). Necessary and sufficient consistency conditions for a recursive kernel regression estimate, J. Multivariate Anal., 23, 67-76.
Greblicki, W., Krzyżak, A. and Pawlak, M. (1984). Distribution-free pointwise consistency of kernel regression estimate, Ann. Statist., 12, 1570-1575.
Györfi, L. (1991). Universal consistencies of a regression estimate for unbounded regression functions, Nonparametric Functional Estimation (ed. G. Roussas), NATO Adv. Sci. Inst. Ser. C Math. Phys. Sci., 329-338, Kluwer, Dordrecht.
Györfi, L. and Walk, H. (1997). On the strong universal consistency of a recursive regression estimate by Pál Révész, Statist. Probab. Lett., 31, 177-183.
Györfi, L., Kohler, M. and Walk, H. (1998). Weak and strong universal consistency of semi-recursive kernel and partitioning regression estimates, Statist. Decisions, 16, 1-18.
Knopp, K. (1956). Infinite Sequences and Series, Dover, New York.
Kozek, A. S., Leslie, J. R. and Schuster, E. F. (1998). On a universal strong law of large numbers for conditional expectations, Bernoulli, 4, 143-165.
Krzyżak, A. (1992). Global convergence of the recursive kernel regression estimates with applications in classification and nonlinear system estimation, IEEE Trans. Inform. Theory, IT-38, 1323-1338.
Krzyżak, A. and Pawlak, M. (1984). Almost everywhere convergence of a recursive regression function estimate and classification, IEEE Trans. Inform. Theory, IT-30, 91-93.
Ljung, L., Pflug, G. and Walk, H. (1992). Stochastic Approximation and Optimization of Random Systems, Birkhäuser, Basel.
Loève, M. (1977). Probability I, 4th ed., Springer, Berlin, Heidelberg, New York.
Nadaraya, E. A. (1964). On estimating regression, Theory Probab. Appl., 9, 141-142.
Révész, P. (1973). Robbins-Monro procedure in a Hilbert space and its application in the theory of learning processes I, Studia Sci. Math. Hungar., 8, 391-398.
Spiegelman, C. and Sacks, J. (1980). Consistent window estimation in nonparametric regression, Ann. Statist., 8, 240-246.
Stone, C. J. (1977). Consistent nonparametric regression, Ann. Statist., 5, 595-645.
Stute, W. (1986). On almost sure convergence of conditional empirical distribution functions, Ann. Probab., 14, 891-901.
Walk, H. (2000). Almost sure convergence properties of Nadaraya-Watson regression estimates, Modeling Uncertainty: An Examination of its Theory, Methods and Applications (S. Yakowitz memorial volume) (eds. M. Dror, P. L'Ecuyer and F. Szidarovszky), Kluwer, Dordrecht (to appear).
Watson, G. S. (1964). Smooth regression analysis, Sankhyā Ser. A, 26, 359-372.
Wheeden, R. L. and Zygmund, A. (1977). Measure and In-0307;tegral, Marcel Dekker, New York.
Wolverton, C. T. and Wagner, T. J. (1969). Recursive estimates of probability densities, IEEE Transactions on Systems, Man, and Cybernetics, 5, 307.
Yamato, H. (1971). Sequential estimation of a continuous probability density function and mode, Bulletin of Mathematical Statistics, 14, 1-12.
Author information
Authors and Affiliations
About this article
Cite this article
Walk, H. Strong Universal Pointwise Consistency of Recursive Regression Estimates. Annals of the Institute of Statistical Mathematics 53, 691–707 (2001). https://doi.org/10.1023/A:1014692616736
Issue Date:
DOI: https://doi.org/10.1023/A:1014692616736