Minimax-rate adaptive nonparametric regression with unknown correlations of errors

  • Guowu Yang
  • Yuhong YangEmail author


Minimax-rate adaptive nonparametric regression has been intensively studied under the assumption of independent or uncorrelated errors in the literature. In many applications, however, the errors are dependent, including both short- and long-range dependent situations. In such a case, adaptation with respect to the unknown dependence is important. We present a general result in this direction under Gaussian errors. It is assumed that the covariance matrix of the errors is known to be in a list of specifications possibly including independence, short-range dependence and long-range dependence as well. The regression function is known to be in a countable (or uncountable but well-structured) collection of function classes. Adaptive estimators are constructed to attain the minimax rate of convergence automatically for each function class under each correlation specification in the corresponding lists.


nonparametric regression adaptive estimation long-range dependence rate of convergence 


62G08 62C20 



This work was supported by National Natural Science Foundation of China (Grant No. 61572109). This paper is dedicated to Professor Lin-Cheng Zhao, in celebration of his 75th birthday. Both authors were inspired by Professor Lincheng Zhao as a leading scholar in statistical sciences. The second author had the honor of being his MS student (co-advised by Professor Baiqi Miao), and received the solid initial training of minimax philosophy and tools. The authors greatly appreciate the helpful comments by two referees on improving their work.


  1. 1.
    Adensdedt R K. On large sample estimation for the mean of a stationary sequence. Ann Statist, 1974, 2: 1095–1107MathSciNetCrossRefGoogle Scholar
  2. 2.
    Baraud Y. Model selection for regression on a random design. ESAIM Probab Stat, 2002, 6: 127–146MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Barron A R. Approximation and estimation bounds for artificial neural networks. Mach Learn, 1994, 14: 115–133zbMATHGoogle Scholar
  4. 4.
    Barron A R, Birgé L, Massart P. Risk bounds for model selection via penalization. Probab Theory Related Fields, 1999, 113: 301–413MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Beran J. Statistics for long-memory processes. Technometrics, 1994, 39: 105–106zbMATHGoogle Scholar
  6. 6.
    Birgé L. Approximation dans les espaces metriques et theorie de l'estimation. Z Wahrscheinlichkeitstheorie Verw Gebiete, 1983, 65: 181–237MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Brown L D, Low M G. A constrained risk inequality with applications to non-parametric functional estimation. Ann Statist, 1996, 24: 2524–2535MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Cover T M, Thomas J A. Elements of Information Theory. Beijing: Tsinghua University Press, 1991CrossRefzbMATHGoogle Scholar
  9. 9.
    Dahlhaus R. Effcient parameter estimation for self-similar processes. Ann Statist, 1989, 17: 1749–1766MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    DeVore R A, Lorentz G G. Constructive Approximation. New York: Springer-Verlag, 1993CrossRefzbMATHGoogle Scholar
  11. 11.
    Donoho D L, Johnstone I M. Minimax estimation via wavelet shrinkage. Ann Statist, 1988, 26: 879–921MathSciNetzbMATHGoogle Scholar
  12. 12.
    Efroimovich Y S, Pinsker M S. A self-educating nonparametric filtration algorithm. Autom Remote Control, 1984, 45: 58–65Google Scholar
  13. 13.
    Efromovich S. How to overcome the curse of long-memory errors. IEEE Trans Inform Theory, 1999, 45: 1735–1741MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Granger C W J, Joyeux R. An introduction to long-range time series models and fractional differencing. J Time Ser Anal, 1980, 1: 15–30MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Hall P, Hart J D. Nonparametric regression with long-range dependence. Stochastic Process Appl, 1990, 36: 339–351MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Hall P, Lahiri S N, Polzehl J. On bandwidth choice in nonparametric regression with both short- and long-range dependent errors. Ann Statist, 1994, 23: 1921–1936MathSciNetzbMATHGoogle Scholar
  17. 17.
    Härdle W, Marron J S. Optimal bandwidth selection in nonparametric regression function estimation. Ann Statist, 1985, 13: 1465–1481MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Horn R, Johnson C R. Matrix Analysis. Cambridge: Cambridge University Press, 1985CrossRefzbMATHGoogle Scholar
  19. 19.
    Hosking J R M. Fractional differencing. Biometrika, 1981, 68: 165–176MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    Johnstone I M, Silverman B W. Wavelet threshold estimators for data with correlated noise. J R Stat Soc Ser B Stat Methodol, 1997, 59: 319–351MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    Kolmogorov A N, Tihomirov V M. ϵ-entropy and ϵ-capacity of sets in functional spaces. Uspekhi Mat Nauk, 1959, 14: 3–86MathSciNetGoogle Scholar
  22. 22.
    Künsch H, Beran J, Hampel F. Contrasts under long-range correlations. Ann Statist, 1993, 21: 943–964MathSciNetCrossRefzbMATHGoogle Scholar
  23. 23.
    LeCam L M. Convergence of estimates under dimensionality restrictions. Ann Statist, 1973, 1: 38–53MathSciNetCrossRefGoogle Scholar
  24. 24.
    Lepskii O V. Asymptotically minimax adaptive estimation I: Upper bounds. Optimally adaptive estimates. Theory Probab Appl, 1991, 36: 682–697CrossRefGoogle Scholar
  25. 25.
    Mandelbrot B B, Van Ness J W. Fractional Brownian motions, fractional noises and applications. SIAM Rev, 1968, 10: 422–437MathSciNetCrossRefzbMATHGoogle Scholar
  26. 26.
    Opsomer J, Wang Y, Yang Y. Nonparametric regression with correlated errors. Statist Sci, 2001, 16: 134–153MathSciNetCrossRefzbMATHGoogle Scholar
  27. 27.
    Samorov A, Taqqu M S. On the effciency of sample mean in long memory noise. J Time Ser Anal, 1988, 9: 191–200MathSciNetCrossRefGoogle Scholar
  28. 28.
    Stone C J. Additive regression and other nonparametric models. Ann Statist, 1985, 13: 689–705MathSciNetCrossRefzbMATHGoogle Scholar
  29. 29.
    Triebel H. Interpolation properties of ϵ-entropy and diameters. Geometric characteristics of imbedding for function spaces of Sobolev-Besov type. Mat Sb, 1975, 98: 27–41Google Scholar
  30. 30.
    Wang Y. Function estimation via wavelet shrinkage for long-memory data. Ann Statist, 1996, 24: 466–484MathSciNetCrossRefzbMATHGoogle Scholar
  31. 31.
    Wang Z, Paterlini S, Gao F, et al. Adaptive minimax regression estimation over sparse ℓq-hulls. J Mach Learn Res, 2014, 15: 1675–1711MathSciNetzbMATHGoogle Scholar
  32. 32.
    Wegkamp M. Model selection in nonparametric regression. Ann Statist, 2003, 31: 252–273MathSciNetCrossRefzbMATHGoogle Scholar
  33. 33.
    Yang Y. Model selection for nonparametric regression. Statist Sinica, 1999, 9: 475–499MathSciNetzbMATHGoogle Scholar
  34. 34.
    Yang Y. Combining different procedures for adaptive regression. J Multivariate Anal, 2000, 74: 135–161MathSciNetCrossRefzbMATHGoogle Scholar
  35. 35.
    Yang Y. Nonparametric regression and prediction with dependent errors. Bernoulli, 2001, 7: 633–655MathSciNetCrossRefzbMATHGoogle Scholar
  36. 36.
    Yang Y. Minimax rate adaptive estimation over continuous hyper-parameters. IEEE Trans Inform Theory, 2001, 47: 2081–2085MathSciNetCrossRefzbMATHGoogle Scholar
  37. 37.
    Yang Y, Barron A R. Information-theoretic determination of minimax rates of convergence. Ann Statist, 1999, 27: 1564–1599MathSciNetCrossRefzbMATHGoogle Scholar
  38. 38.
    Zhao L C. Exponential bounds of mean error for the nearest neighbor estimates of regression functions. J Multivariate Anal, 1987, 21: 168–178MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Science China Press and Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.School of Computer Science and EngineeringUniversity of Electronic Science and Technology of ChinaChengduChina
  2. 2.School of StatisticsUniversity of MinnesotaMinneapolisUSA

Personalised recommendations