Abstract
We consider a nonparametric regression setup, where the covariate is a random element in a complete separable metric space, and the parameter of interest associated with the conditional distribution of the response lies in a separable Banach space. We derive the optimum convergence rate for the kernel estimate of the parameter in this setup. The small ball probability in the covariate space plays a critical role in determining the asymptotic variance of kernel estimates. Unlike the case of finite-dimensional covariates, we show that the asymptotic orders of the bias and the variance of the estimate achieving the optimum convergence rate may be different for infinite-dimensional covariates. Also, the bandwidth, which balances the bias and the variance, may lead to an estimate with suboptimal mean square error for infinite-dimensional covariates. We describe a data-driven adaptive choice of the bandwidth and derive the asymptotic behavior of the adaptive estimate.
This is a preview of subscription content, access via your institution.







References
Aerts, M., Claeskens, G. (1997). Local polynomial estimation in multiparameter likelihood models. Journal of the American Statistical Association, 92(440), 1536–1545.
Araujo, A., Giné, E. (1980). The central limit theorem for real and Banach valued random variables. New York: Wiley.
Bhatia, R. (2009). Notes on functional analysis. New Delhi: Hindustan Book Agency.
Burba, F., Ferraty, F., Vieu, P. (2009). k-Nearest neighbour method in functional nonparametric regression. Journal of Nonparametric Statistics, 21(4), 453–469.
Cameron, R. H., Martin, W. T. (1944). Transformations of Weiner integrals under translations. Annals of Mathematics, 45(2), 386–396.
Chagny, G., Roche, A. (2014). Adaptive and minimax estimation of the cumulative distribution function given a functional covariate. Electronic Journal of Statistics, 8(2), 2352–2404.
Chagny, G., Roche, A. (2016). Adaptive estimation in the functional nonparametric regression model. Journal of Multivariate Analysis, 146, 105–118.
Chaouch, M., Laïb, N. (2013). Nonparametric multivariate $ \text{ L }_1 $-median regression estimation with functional covariates. Electronic Journal of Statistics, 7, 1553–1586.
Chaouch, M., Laïb, N. (2015). Vector-on-function quantile regression for stationary ergodic processes. Journal of the Korean Statistical Society, 44(2), 161–178.
Chaudhuri, P., Dewanji, A. (1995). On a likelihood-based approach in nonparametric smoothing and cross-validation. Statistics & Probability Letters, 22(1), 7–15.
Dereich, S., Lifshits, M. (2005). Probabilities of randomly centered small balls and quantization in Banach spaces. The Annals of Probability, 33(4), 1397–1421.
Dette, H., Wieczorek, G. (2009). Testing for a constant coefficient of variation in nonparametric regression. Journal of Statistical Theory and Practice, 3(3), 587–612.
Dette, H., Marchlewski, M., Wagener, J. (2012). Testing for a constant coefficient of variation in nonparametric regression by empirical processes. Annals of the Institute of Statistical Mathematics, 64(5), 1045–1070.
Donoho, D. L., Liu, R. C. (1991a). Geometrizing rates of convergence, II. The Annals of Statistics, 19(2), 633–667.
Donoho, D. L., Liu, R. C. (1991b). Geometrizing rates of convergence, III. The Annals of Statistics, 19(2), 668–701.
Ferraty, F., Vieu, P. (2006). Nonparametric functional data analysis: Theory and practice. New York: Springer.
Ferraty, F., Laksaci, A., Vieu, P. (2006). Estimating some characteristics of the conditional distribution in nonparametric functional models. Statistical Inference for Stochastic Processes, 9(1), 47–76.
Ferraty, F., Mas, A., Vieu, P. (2007). Nonparametric regression on functional data: Inference and practical aspects. Australian & New Zealand Journal of Statistics, 49(3), 267–286.
Ferraty, F., Laksaci, A., Tadj, A., Vieu, P. (2010). Rate of uniform consistency for nonparametric estimates with functional variables. Journal of Statistical Planning and Inference, 140(2), 335–352.
Ferraty, F., Park, J., Vieu, P. (2011). Estimation of a functional single index model. In F. Ferraty (Ed.), Recent advances in functional data analysis and related topics, chapter 17, pp. 111–116. New York: Springer.
Ferraty, F., Van Keilegom, I., Vieu, P. (2012). Regression when both response and predictor are functions. Journal of Multivariate Analysis, 109, 10–28.
Ferré, L., Yao, A. (2005). Smoothed functional inverse regression. Statistica Sinica, 15(3), 665.
Ferré, L., Yao, A.-F. (2003). Functional sliced inverse regression analysis. Statistics, 37(6), 475–488.
Hardle, W. (1990). Applied nonparametric regression. Cambridge: Cambridge University Press.
Hoffmann-Jorgensen, J., Shepp, L. A., Dudley, R. M. (1979). On the lower tail of Gaussian seminorms. The Annals of Probability, 7(2), 319–342.
Ibragimov, I. A., Haśminskii, R. Z. (1980). On nonparametric estimation of regression. Soviet Mathematics Doklady, 21, 810–814.
Klemelä, J. S. (2014). Multivariate nonparametric regression and visualization: With R and applications to finance. Hoboken: Wiley.
Kundu, S., Majumdar, S., Mukherjee, K. (2000). Central limit theorems revisited. Statistics and Probability Letters, 47(3), 265–275.
Li, W. V. (2001). Small ball probabilities for Gaussian Markov processes under the $ \text{ L }_p $-norm. Stochastic Processes and Their Applications, 92(1), 87–102.
Li, W. V., Shao, Q.-M. (2001). Gaussian processes: Inequalities, small ball probabilities and applications. Stochastic Processes: Theory and Methods, 19, 533–597.
Lian, H. (2012). Convergence of nonparametric functional regression estimates with functional responses. Electronic Journal of Statistics, 6, 1373–1391.
Lifshits, M. A. (2013). Gaussian random functions, Vol. 322. Dordrecht: Springer.
Lukić, M., Beder, J. (2001). Stochastic processes with sample paths in reproducing kernel Hilbert spaces. Transactions of the American Mathematical Society, 353(10), 3945–3969.
Mas, A. (2012). Lower bound in regression for functional data by representation of small ball probabilities. Electronic Journal of Statistics, 6, 1745–1778.
Masry, E. (2005). Nonparametric regression estimation for dependent functional data: Asymptotic normality. Stochastic Processes and Their Applications, 115(1), 155–177.
Nadaraya, E. A. (1964). On estimating regression. Theory of Probability and Its Applications, 9(1), 141–142.
Øksendal, B. (2003). Stochastic differential equations: An introduction with applications. New York: Springer.
Rachdi, M., Vieu, P. (2007). Nonparametric regression for functional data: Automatic smoothing parameter selection. Journal of Statistical Planning and Inference, 137(9), 2784–2801.
Serfling, R. J. (2009). Approximation theorems of mathematical statistics, Vol. 162. Hoboken: Wiley.
Staniswalis, J. G. (1989). The kernel estimate of a regression function in likelihood-based models. Journal of the American Statistical Association, 84(405), 276–283.
Stone, C. J. (1980). Optimal rates of convergence for nonparametric estimators. The Annals of Statistics, 8(6), 1348–1360.
Stone, C. J. (1982). Optimal global rates of convergence for nonparametric regression. The Annals of Statistics, 10(4), 1040–1053.
Vepakomma, P., Tonde, C., Elgammal, A. (2016). Supervised dimensionality reduction via distance correlation maximization. arXiv preprint arXiv:1601.00236.
Watson, G. S. (1964). Smooth regression analysis. Sankhyā: The Indian Journal of Statistics, Series A, 26(4), 359–372.
Yatracos, Y. G. (1988). A lower bound on the error in nonparametric regression type problems. The Annals of Statistics, 16(3), 1180–1187.
Acknowledgements
We thank the Editor, the Associate Editor and three reviewers for their extremely careful reading and valuable comments and suggestions that led to a substantially revised and significantly improved version of the paper.
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
About this article
Cite this article
Chowdhury, J., Chaudhuri, P. Convergence rates for kernel regression in infinite-dimensional spaces. Ann Inst Stat Math 72, 471–509 (2020). https://doi.org/10.1007/s10463-018-0697-2
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10463-018-0697-2
Keywords
- Adaptive estimate
- Bias-variance decomposition
- Gaussian process
- Maximum likelihood regression
- Mean square error
- Optimal bandwidth
- Small ball probability
- t process