Skip to main content
SpringerLink
Log in

The reproducing kernel Hilbert space approach in nonparametric regression problems with correlated observations

  • Published:
Annals of the Institute of Statistical Mathematics Aims and scope Submit manuscript

Abstract

In this paper, we investigate the problem of estimating the regression function in models with correlated observations. The data are obtained from several experimental units, each of them forms a time series. Using the properties of the reproducing kernel Hilbert spaces, we construct a new estimator based on the inverse of the autocovariance matrix of the observations. We give the asymptotic expressions of its bias and its variance. In addition, we give a theoretical comparison between this new estimator and the popular one proposed by Gasser and Müller, we show that the proposed estimator has an asymptotically smaller variance than the classical one. Finally, we conduct a simulation study to investigate the performance and the robustness of the proposed estimator and to compare it to the Gasser and Müller’s estimator in a finite sample set.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Notes

  1. The kernel K satisfies: \(\int _{-1}^1 K(t)\hbox {d}t=1,\) \(\int _{-1}^1 t K(t)\hbox {d}t=0\) and \(\int _{-1}^1 t^2 K(t)\hbox {d}t <+ \infty .\)

References

  • Altman, N. S. (1990). Kernel smoothing of data with correlated errors. American Statistical Association, 85, 749–759.

    Article  MathSciNet  Google Scholar 

  • Azzalini, A. (1984). Estimation and hypothesis testing for collections of autoregressive time series. Biometrika, 71, 85–90.

    Article  MathSciNet  Google Scholar 

  • Belouni, M., Benhenni, K. (2015). Optimal and robust designs for estimating the concentration curve and the AUC. Scandinavian Journal of Statistics: Theory and Application, 42, 453–470.

  • Benedetti, J. (1977). On the nonparametric estimation of the regression function. Journal of the Royal Statistical Society, 39, 248–253.

    MathSciNet  MATH  Google Scholar 

  • Benelmadani, D. (2019a). Contribution à la Régression non Paramétrique avec un Processus Erreur d’Autocovariance Générale et Application en Pharmacocinétique, Ph.D. 2019, Grenoble Alpes University, France.

  • Benelmadani, D., Benhenni, K., Louhichi, S. (2019b). Trapezoidal rule and sampling designs for the nonparametric estimation of the regression function in models with correlated errors. arXiv:1806.04896.

  • Benhenni, K., Cambanis, S. (1992). Sampling designs for estimating integrals of stochastic processes. The Annals of Statistics, 20, 161–194.

    Article  MathSciNet  Google Scholar 

  • Benhenni, K., Rachdi, M. (2007). Nonparametric estimation of average growth curve with general nonstationary error process. Communications in Statistics: Theory and Methods, 36, 1137–1186.

  • Blanke, D., Bosq, D. (2008). Regression estimation and prediction in continuous time. Journal of the Japan Statistical Society (Nihon Tôkei Gakkai Kaihô), 38, 15–26.

    Article  MathSciNet  Google Scholar 

  • Cheng, K. F., Lin, P. E. (1981). Nonparametric estimation of the regression function. Z. Wahrscheinlichkeitstheorie verw. Gebiete, 57, 223–233.

    Article  MathSciNet  Google Scholar 

  • Chiu, S. T. (1989). Bandwidth selection for kernel estimation with correlated noise. Statistics and Probability Letters, 8, 347–354.

    Article  MathSciNet  Google Scholar 

  • Dette, H., Pepelyshev, A., Zhigljavsky, A. (2016). Best linear unbiased estimators in continuous time regression models. arXiv:1611.09804.

  • Didi, S., Louani, D. (2013). Asymptotic results for the regression function estimate on continuous time stationary and ergodic data. Journal of Statistics and Risk Modelling, 31(2), 129–150.

    MathSciNet  MATH  Google Scholar 

  • Ferreira, E., Núǹez-Antón, V., Rodríguez-Póo, J. (1997). Kernel regression estimates of growth curves using nonstationary correlated errors. Statistics and Probability Letters, 34, 413–423.

    Article  MathSciNet  Google Scholar 

  • Gasser, T., Müller, H. G. (1979). Kernel estimation of regression functions. Lecture Notes in Mathematics, 757, 23–68.

    Article  MathSciNet  Google Scholar 

  • Hart, J. D. (1991). Kernel regression estimation with time series errors. Royal Statistical Society B, 53, 173–187.

    MathSciNet  MATH  Google Scholar 

  • Hart, J. D. (1994). Automated kernel smoothing of dependent data by using time series cross validation. Royal Statistical Society B, 56, 529–542.

    MathSciNet  MATH  Google Scholar 

  • Hart, J. D., Wherly, T. E. (1986). Kernel regression estimation using repeated measurements data. Journal of the American Statistical Association, 81, 1080–1088.

    Article  MathSciNet  Google Scholar 

  • Müller, G. H. (1984). Optimal designs for nonparametric kernel regression. Statistics and Probability Letters, 2, 285–290.

    Article  MathSciNet  Google Scholar 

  • Parzen, E. (1959). Statistical inference on time series by hilbert space methods, p. 23. Technical Report, Department of Statistics, Stanford University, Stanford, CA.

  • Priestly, M. B., Chao, M. T. (1972). Nonparametric function fitting. Journal of Royal Statistical Society, 34, 384–392.

    Google Scholar 

  • Ramsay, J. O., Silverman, B. W. (2005). Functional data analysis, Springer series in statistics, New York: Springer.

  • Rice, J. (1984). Bandwidth choice for nonparametric regression. The Annals of Statistics, 12, 1215–1230.

    Article  MathSciNet  Google Scholar 

  • Sacks, J., Ylvisaker, D. (1966). Designs for regression problems with correlated errors. The Annals of Mathematical Statistics, 37, 66–89.

    Article  MathSciNet  Google Scholar 

  • Sacks, J., Ylvisaker, D. (1968). Designs for regression problems with correlated errors: Many parameters. The Annals of Mathematical Statistics, 39, 69–79.

    Article  MathSciNet  Google Scholar 

  • Sacks, J., Ylvisaker, D. (1970). Designs for regression problems with correlated errors III. The Annals of Mathematical Statistics, 41, 2057–2074.

    Article  MathSciNet  Google Scholar 

  • Schwartz, L. (1964). Sous Espace Hilbertiens d’Espace Vectoriels Topologiques et Noyaux Associés (Noyaux Reproduisant). Journal d’Analyse Mathématique, 13, 115–256.

    Article  MathSciNet  Google Scholar 

  • Su, Y., Cambanis, S. (1993). Sampling designs for estimation of a random process. Stochastic Processes and their Applications, 46, 47–89.

    Article  MathSciNet  Google Scholar 

  • Xiang, Y., Gubian, S., Suomela, B., Hoeng, J. (2013). Generalized simulated annealing for efficient global optimization: The GenSA package for R. The R Journal, 5, 13–28.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Laboratoire Jean Kuntzmann (CNRS 5224), Université Grenoble Alpes, 700 Avenue Centrale, 38401, Saint-Martin-d’Hères, France

    D. Benelmadani, K. Benhenni & S. Louhichi

Authors
  1. D. Benelmadani
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. K. Benhenni
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. S. Louhichi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to K. Benhenni.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 441 KB)

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Benelmadani, D., Benhenni, K. & Louhichi, S. The reproducing kernel Hilbert space approach in nonparametric regression problems with correlated observations. Ann Inst Stat Math 72, 1479–1500 (2020). https://doi.org/10.1007/s10463-019-00733-3

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10463-019-00733-3

Keywords

Search

Navigation