Computational Statistics

, Volume 33, Issue 2, pp 863–885 | Cite as

Optimal difference-based estimation for partially linear models

Original Paper
  • 91 Downloads

Abstract

Difference-based methods have attracted increasing attention for analyzing partially linear models in the recent literature. In this paper, we first propose to solve the optimal sequence selection problem in difference-based estimation for the linear component. To achieve the goal, a family of new sequences and a cross-validation method for selecting the adaptive sequence are proposed. We demonstrate that the existing sequences are only extreme cases in the proposed family. Secondly, we propose a new estimator for the residual variance by fitting a linear regression method to some difference-based estimators. Our proposed estimator achieves the asymptotic optimal rate of mean squared error. Simulation studies also demonstrate that our proposed estimator performs better than the existing estimator, especially when the sample size is small and the nonparametric function is rough.

Keywords

Asymptotic normality Difference-based method Difference sequence Least squares estimator Partially linear model 

Notes

Acknowledgements

Yuejin Zhou’s research was supported in part by the Natural Science Foundation of Anhui Grant (No. KJ2017A087), and the National Natural Science Foundation of China Grant (No. 61472003). Yebin Cheng’s research was supported in part by the National Natural Science Foundation of China Grant (No. 11271241). Tiejun Tong’s research was supported in part by the Hong Kong Baptist University Grants FRG1/16-17/018 and FRG2/16-17/074, and the National Natural Science Foundation of China Grant (No. 11671338).

References

  1. Akdeniz F, Duran E (2013) New difference-based estimator of parameters in semiparametric regression models. J Stat Comput Simul 83:810–824MathSciNetCrossRefMATHGoogle Scholar
  2. Aneiros G, Ling N, Vieu P (2015) Error variance estimation in semi-functional partially linear regression models. J Nonparametr Stat 27:316–330MathSciNetCrossRefMATHGoogle Scholar
  3. Chen H, Shiau J (1991) A two-stage spline smoothing method for partially linear models. J Stat Plan Inference 27:187–201MathSciNetCrossRefMATHGoogle Scholar
  4. Cuzick J (1992) Semiparametric additive regression. J Roy Stat Soc B 54:831–843MathSciNetMATHGoogle Scholar
  5. Dai W, Ma Y, Tong T, Zhu L (2015) Difference-based variance estimation in nonparametric regression with repeated measurement data. J Stat Plan Inference 163:1–20MathSciNetCrossRefMATHGoogle Scholar
  6. Dai W, Tong T, Genton M (2016) Optimal estimation of derivatives in nonparametric regression. J Mach Learn Res 17:1–25MathSciNetMATHGoogle Scholar
  7. Dai W, Tong T, Zhu L (2017) On the choice of difference sequence in a unified framework for variance estimation in nonparametric regression. Stat Sci 32:455–468MathSciNetCrossRefGoogle Scholar
  8. Dette H, Munk A, Wagner T (1998) Estimating the variance in nonparametric regression—what is a reasonable choice? J Roy Stat Soc B 60:751–764MathSciNetCrossRefMATHGoogle Scholar
  9. Eubank R, Kambour E, Kim J, Klipple K, Reese C, Schimek M (1998) Estimation in partially linear models. Comput Stat Data Anal 29:27–34MathSciNetCrossRefMATHGoogle Scholar
  10. Fan J, Huang T (2005) Profile likelihood inferences on semiparametric varying-coefficient partially linear models. Bernoulli 11:1031–1057MathSciNetCrossRefMATHGoogle Scholar
  11. Gasser T, Sroka L, Jennen-Steinmetz C (1986) Residual variance and residual pattern in nonlinear regression. Biometrika 73:625–633MathSciNetCrossRefMATHGoogle Scholar
  12. Hall P, Kay JW, Titterington DM (1990) Asymptotically optimal difference-based estimation of variance in nonparametric regression. Biometrika 77:521–528MathSciNetCrossRefMATHGoogle Scholar
  13. Hall P, Keilegom IV (2003) Using difference-based methods for inference in nonparametric regression with time series errors. J Roy Stat Soc B 65:443–456MathSciNetCrossRefMATHGoogle Scholar
  14. Hardle W, Liang H, Gao J (2000) Partially linear models. Physika Verlag, HeidelbergCrossRefMATHGoogle Scholar
  15. He H, Tang W, Zuo G (2014) Statistical inference in the partial inear models with the doubles moothing local linear regression method. J Stat Plan Inference 146:102–112CrossRefGoogle Scholar
  16. Hu H, Zhang Y, Pan X (2016) Asymptotic normality of dhd estimators in a partially linear model. Stat Pap 57:567–587MathSciNetCrossRefMATHGoogle Scholar
  17. Levine M (2015) Minimax rate of convergence for an estimator of the functional component in a semiparametric multivariate partially linear model. J Multivar Anal 140:283–290MathSciNetCrossRefMATHGoogle Scholar
  18. Liu Q, Zhao G (2012) A comparison of estimation methods for partially linear models. Int J Innov Manag Inf Prod 3:38–42Google Scholar
  19. Lokshin M (2006) Difference-based semiparametric estimation of partial linear regression models. Stata J 6:377–383Google Scholar
  20. Rice JA (1984) Bandwidth choice for nonparametric regression. Ann Stat 12:1215–1230MathSciNetCrossRefMATHGoogle Scholar
  21. Schott JR (1997) Matrix analysis for statistics. Wiley, New YorkMATHGoogle Scholar
  22. Severini T, Wong W (1992) Generalized profile likelihhood and conditional parametric models. Ann Stat 20:1768–1802CrossRefMATHGoogle Scholar
  23. Spechman P (1988) Kernel smoothing in partial linear models. J Roy Stat Soc B 50:413–436MathSciNetGoogle Scholar
  24. Tabakan G (2013) Performance of the difference-based estimators in partially linear models. Statistics 47:329–347MathSciNetCrossRefMATHGoogle Scholar
  25. Tong T, Ma Y, Wang Y (2013) Optimal variance estimation without estimating the meam function. Bernoulli 19:1839–1854MathSciNetCrossRefMATHGoogle Scholar
  26. Tong T, Wang Y (2005) Estimating residual variance in nonparametric regression using least squares. Biometrika 92:821–830MathSciNetCrossRefMATHGoogle Scholar
  27. Wang L, Brown L, Cai T (2011) A difference based approach to the semiparametric partial linear model. Electron J Stat 5:619–641MathSciNetCrossRefMATHGoogle Scholar
  28. Wang W, Lin L (2015) Derivative estimation based on difference sequence via locally weighted least squares regession. J Mach Learn Res 16:2617–2641MathSciNetMATHGoogle Scholar
  29. Wang W, Lin L, Yu L (2017) Optimal variance estimation based on lagged second-order difference in nonparametric regression. Comput Stat 32:1047–1063MathSciNetCrossRefMATHGoogle Scholar
  30. Whittle P (1964) On the convergence to normality of quadratic forms in independent variables. Teor. Verojatnost. i Primenen 9:113–118MathSciNetMATHGoogle Scholar
  31. Yatchew A (1997) An elementary estimator of the partial linear model. Econ Lett 57:135–143MathSciNetCrossRefMATHGoogle Scholar
  32. Zhao H, You J (2011) Difference based estimation for partially linear regression models with measurement errors. J Multivar Anal 102:1321–1338MathSciNetCrossRefMATHGoogle Scholar
  33. Zhou Y, Cheng Y, Wang L, Tong T (2015) Optimal difference-based variance estimation in heteroscedastic nonparametric regression. Stat Sin 25:1377–1397MathSciNetMATHGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2017

Authors and Affiliations

  1. 1.School of Mathematics and Big DataAnhui University of Science and TechnologyHuainanChina
  2. 2.School of Statistics and MathematicsZhejiang Gongshang UniversityHangzhouChina
  3. 3.Glorious Sun School of Business and ManagementDonghua UniversityShanghaiChina
  4. 4.CEMSE DivisionKing Abdullah University of Science and TechnologyThuwalSaudi Arabia
  5. 5.Department of MathematicsHong Kong Baptist UniversityKowloonChina

Personalised recommendations