Skip to main content
SpringerLink
Log in

Convergence rates for kernel regression in infinite-dimensional spaces

  • Published:
Annals of the Institute of Statistical Mathematics Aims and scope Submit manuscript

Abstract

We consider a nonparametric regression setup, where the covariate is a random element in a complete separable metric space, and the parameter of interest associated with the conditional distribution of the response lies in a separable Banach space. We derive the optimum convergence rate for the kernel estimate of the parameter in this setup. The small ball probability in the covariate space plays a critical role in determining the asymptotic variance of kernel estimates. Unlike the case of finite-dimensional covariates, we show that the asymptotic orders of the bias and the variance of the estimate achieving the optimum convergence rate may be different for infinite-dimensional covariates. Also, the bandwidth, which balances the bias and the variance, may lead to an estimate with suboptimal mean square error for infinite-dimensional covariates. We describe a data-driven adaptive choice of the bandwidth and derive the asymptotic behavior of the adaptive estimate.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  • Aerts, M., Claeskens, G. (1997). Local polynomial estimation in multiparameter likelihood models. Journal of the American Statistical Association, 92(440), 1536–1545.

    Article  MathSciNet  MATH  Google Scholar 

  • Araujo, A., Giné, E. (1980). The central limit theorem for real and Banach valued random variables. New York: Wiley.

    MATH  Google Scholar 

  • Bhatia, R. (2009). Notes on functional analysis. New Delhi: Hindustan Book Agency.

    Book  MATH  Google Scholar 

  • Burba, F., Ferraty, F., Vieu, P. (2009). k-Nearest neighbour method in functional nonparametric regression. Journal of Nonparametric Statistics, 21(4), 453–469.

    Article  MathSciNet  MATH  Google Scholar 

  • Cameron, R. H., Martin, W. T. (1944). Transformations of Weiner integrals under translations. Annals of Mathematics, 45(2), 386–396.

    Article  MathSciNet  MATH  Google Scholar 

  • Chagny, G., Roche, A. (2014). Adaptive and minimax estimation of the cumulative distribution function given a functional covariate. Electronic Journal of Statistics, 8(2), 2352–2404.

    Article  MathSciNet  MATH  Google Scholar 

  • Chagny, G., Roche, A. (2016). Adaptive estimation in the functional nonparametric regression model. Journal of Multivariate Analysis, 146, 105–118.

    Article  MathSciNet  MATH  Google Scholar 

  • Chaouch, M., Laïb, N. (2013). Nonparametric multivariate $ \text{ L }_1 $-median regression estimation with functional covariates. Electronic Journal of Statistics, 7, 1553–1586.

    Article  MathSciNet  MATH  Google Scholar 

  • Chaouch, M., Laïb, N. (2015). Vector-on-function quantile regression for stationary ergodic processes. Journal of the Korean Statistical Society, 44(2), 161–178.

    Article  MathSciNet  MATH  Google Scholar 

  • Chaudhuri, P., Dewanji, A. (1995). On a likelihood-based approach in nonparametric smoothing and cross-validation. Statistics & Probability Letters, 22(1), 7–15.

    Article  MathSciNet  MATH  Google Scholar 

  • Dereich, S., Lifshits, M. (2005). Probabilities of randomly centered small balls and quantization in Banach spaces. The Annals of Probability, 33(4), 1397–1421.

    Article  MathSciNet  MATH  Google Scholar 

  • Dette, H., Wieczorek, G. (2009). Testing for a constant coefficient of variation in nonparametric regression. Journal of Statistical Theory and Practice, 3(3), 587–612.

    Article  MathSciNet  MATH  Google Scholar 

  • Dette, H., Marchlewski, M., Wagener, J. (2012). Testing for a constant coefficient of variation in nonparametric regression by empirical processes. Annals of the Institute of Statistical Mathematics, 64(5), 1045–1070.

    Article  MathSciNet  MATH  Google Scholar 

  • Donoho, D. L., Liu, R. C. (1991a). Geometrizing rates of convergence, II. The Annals of Statistics, 19(2), 633–667.

    Article  MathSciNet  MATH  Google Scholar 

  • Donoho, D. L., Liu, R. C. (1991b). Geometrizing rates of convergence, III. The Annals of Statistics, 19(2), 668–701.

    Article  MathSciNet  MATH  Google Scholar 

  • Ferraty, F., Vieu, P. (2006). Nonparametric functional data analysis: Theory and practice. New York: Springer.

  • Ferraty, F., Laksaci, A., Vieu, P. (2006). Estimating some characteristics of the conditional distribution in nonparametric functional models. Statistical Inference for Stochastic Processes, 9(1), 47–76.

    Article  MathSciNet  MATH  Google Scholar 

  • Ferraty, F., Mas, A., Vieu, P. (2007). Nonparametric regression on functional data: Inference and practical aspects. Australian & New Zealand Journal of Statistics, 49(3), 267–286.

    Article  MathSciNet  MATH  Google Scholar 

  • Ferraty, F., Laksaci, A., Tadj, A., Vieu, P. (2010). Rate of uniform consistency for nonparametric estimates with functional variables. Journal of Statistical Planning and Inference, 140(2), 335–352.

    Article  MathSciNet  MATH  Google Scholar 

  • Ferraty, F., Park, J., Vieu, P. (2011). Estimation of a functional single index model. In F. Ferraty (Ed.), Recent advances in functional data analysis and related topics, chapter 17, pp. 111–116. New York: Springer.

    Chapter  Google Scholar 

  • Ferraty, F., Van Keilegom, I., Vieu, P. (2012). Regression when both response and predictor are functions. Journal of Multivariate Analysis, 109, 10–28.

    Article  MathSciNet  MATH  Google Scholar 

  • Ferré, L., Yao, A. (2005). Smoothed functional inverse regression. Statistica Sinica, 15(3), 665.

    MathSciNet  MATH  Google Scholar 

  • Ferré, L., Yao, A.-F. (2003). Functional sliced inverse regression analysis. Statistics, 37(6), 475–488.

    Article  MathSciNet  MATH  Google Scholar 

  • Hardle, W. (1990). Applied nonparametric regression. Cambridge: Cambridge University Press.

    Book  MATH  Google Scholar 

  • Hoffmann-Jorgensen, J., Shepp, L. A., Dudley, R. M. (1979). On the lower tail of Gaussian seminorms. The Annals of Probability, 7(2), 319–342.

    Article  MathSciNet  MATH  Google Scholar 

  • Ibragimov, I. A., Haśminskii, R. Z. (1980). On nonparametric estimation of regression. Soviet Mathematics Doklady, 21, 810–814.

    Google Scholar 

  • Klemelä, J. S. (2014). Multivariate nonparametric regression and visualization: With R and applications to finance. Hoboken: Wiley.

  • Kundu, S., Majumdar, S., Mukherjee, K. (2000). Central limit theorems revisited. Statistics and Probability Letters, 47(3), 265–275.

    Article  MathSciNet  MATH  Google Scholar 

  • Li, W. V. (2001). Small ball probabilities for Gaussian Markov processes under the $ \text{ L }_p $-norm. Stochastic Processes and Their Applications, 92(1), 87–102.

    Article  MathSciNet  Google Scholar 

  • Li, W. V., Shao, Q.-M. (2001). Gaussian processes: Inequalities, small ball probabilities and applications. Stochastic Processes: Theory and Methods, 19, 533–597.

  • Lian, H. (2012). Convergence of nonparametric functional regression estimates with functional responses. Electronic Journal of Statistics, 6, 1373–1391.

    Article  MathSciNet  MATH  Google Scholar 

  • Lifshits, M. A. (2013). Gaussian random functions, Vol. 322. Dordrecht: Springer.

    Google Scholar 

  • Lukić, M., Beder, J. (2001). Stochastic processes with sample paths in reproducing kernel Hilbert spaces. Transactions of the American Mathematical Society, 353(10), 3945–3969.

    Article  MathSciNet  MATH  Google Scholar 

  • Mas, A. (2012). Lower bound in regression for functional data by representation of small ball probabilities. Electronic Journal of Statistics, 6, 1745–1778.

    Article  MathSciNet  MATH  Google Scholar 

  • Masry, E. (2005). Nonparametric regression estimation for dependent functional data: Asymptotic normality. Stochastic Processes and Their Applications, 115(1), 155–177.

    Article  MathSciNet  MATH  Google Scholar 

  • Nadaraya, E. A. (1964). On estimating regression. Theory of Probability and Its Applications, 9(1), 141–142.

    Article  MATH  Google Scholar 

  • Øksendal, B. (2003). Stochastic differential equations: An introduction with applications. New York: Springer.

  • Rachdi, M., Vieu, P. (2007). Nonparametric regression for functional data: Automatic smoothing parameter selection. Journal of Statistical Planning and Inference, 137(9), 2784–2801.

    Article  MathSciNet  MATH  Google Scholar 

  • Serfling, R. J. (2009). Approximation theorems of mathematical statistics, Vol. 162. Hoboken: Wiley.

    MATH  Google Scholar 

  • Staniswalis, J. G. (1989). The kernel estimate of a regression function in likelihood-based models. Journal of the American Statistical Association, 84(405), 276–283.

    Article  MathSciNet  MATH  Google Scholar 

  • Stone, C. J. (1980). Optimal rates of convergence for nonparametric estimators. The Annals of Statistics, 8(6), 1348–1360.

    Article  MathSciNet  MATH  Google Scholar 

  • Stone, C. J. (1982). Optimal global rates of convergence for nonparametric regression. The Annals of Statistics, 10(4), 1040–1053.

    Article  MathSciNet  MATH  Google Scholar 

  • Vepakomma, P., Tonde, C., Elgammal, A. (2016). Supervised dimensionality reduction via distance correlation maximization. arXiv preprint arXiv:1601.00236.

  • Watson, G. S. (1964). Smooth regression analysis. Sankhyā: The Indian Journal of Statistics, Series A, 26(4), 359–372.

  • Yatracos, Y. G. (1988). A lower bound on the error in nonparametric regression type problems. The Annals of Statistics, 16(3), 1180–1187.

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

We thank the Editor, the Associate Editor and three reviewers for their extremely careful reading and valuable comments and suggestions that led to a substantially revised and significantly improved version of the paper.

Author information

Authors and Affiliations

  1. Statistics and Mathematics Unit, Indian Statistical Institute, 203, B.T. Road, Kolkata, 700108, India

    Joydeep Chowdhury & Probal Chaudhuri

Authors
  1. Joydeep Chowdhury
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Probal Chaudhuri
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Joydeep Chowdhury.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 316 KB)

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chowdhury, J., Chaudhuri, P. Convergence rates for kernel regression in infinite-dimensional spaces. Ann Inst Stat Math 72, 471–509 (2020). https://doi.org/10.1007/s10463-018-0697-2

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10463-018-0697-2

Keywords

Search

Navigation