Computational Statistics

, Volume 33, Issue 2, pp 863–885 | Cite as

Optimal difference-based estimation for partially linear models

  • Yuejin Zhou
  • Yebin Cheng
  • Wenlin Dai
  • Tiejun Tong
Original Paper


Difference-based methods have attracted increasing attention for analyzing partially linear models in the recent literature. In this paper, we first propose to solve the optimal sequence selection problem in difference-based estimation for the linear component. To achieve the goal, a family of new sequences and a cross-validation method for selecting the adaptive sequence are proposed. We demonstrate that the existing sequences are only extreme cases in the proposed family. Secondly, we propose a new estimator for the residual variance by fitting a linear regression method to some difference-based estimators. Our proposed estimator achieves the asymptotic optimal rate of mean squared error. Simulation studies also demonstrate that our proposed estimator performs better than the existing estimator, especially when the sample size is small and the nonparametric function is rough.


Asymptotic normality Difference-based method Difference sequence Least squares estimator Partially linear model 



Yuejin Zhou’s research was supported in part by the Natural Science Foundation of Anhui Grant (No. KJ2017A087), and the National Natural Science Foundation of China Grant (No. 61472003). Yebin Cheng’s research was supported in part by the National Natural Science Foundation of China Grant (No. 11271241). Tiejun Tong’s research was supported in part by the Hong Kong Baptist University Grants FRG1/16-17/018 and FRG2/16-17/074, and the National Natural Science Foundation of China Grant (No. 11671338).


  1. Akdeniz F, Duran E (2013) New difference-based estimator of parameters in semiparametric regression models. J Stat Comput Simul 83:810–824MathSciNetCrossRefzbMATHGoogle Scholar
  2. Aneiros G, Ling N, Vieu P (2015) Error variance estimation in semi-functional partially linear regression models. J Nonparametr Stat 27:316–330MathSciNetCrossRefzbMATHGoogle Scholar
  3. Chen H, Shiau J (1991) A two-stage spline smoothing method for partially linear models. J Stat Plan Inference 27:187–201MathSciNetCrossRefzbMATHGoogle Scholar
  4. Cuzick J (1992) Semiparametric additive regression. J Roy Stat Soc B 54:831–843MathSciNetzbMATHGoogle Scholar
  5. Dai W, Ma Y, Tong T, Zhu L (2015) Difference-based variance estimation in nonparametric regression with repeated measurement data. J Stat Plan Inference 163:1–20MathSciNetCrossRefzbMATHGoogle Scholar
  6. Dai W, Tong T, Genton M (2016) Optimal estimation of derivatives in nonparametric regression. J Mach Learn Res 17:1–25MathSciNetzbMATHGoogle Scholar
  7. Dai W, Tong T, Zhu L (2017) On the choice of difference sequence in a unified framework for variance estimation in nonparametric regression. Stat Sci 32:455–468MathSciNetCrossRefGoogle Scholar
  8. Dette H, Munk A, Wagner T (1998) Estimating the variance in nonparametric regression—what is a reasonable choice? J Roy Stat Soc B 60:751–764MathSciNetCrossRefzbMATHGoogle Scholar
  9. Eubank R, Kambour E, Kim J, Klipple K, Reese C, Schimek M (1998) Estimation in partially linear models. Comput Stat Data Anal 29:27–34MathSciNetCrossRefzbMATHGoogle Scholar
  10. Fan J, Huang T (2005) Profile likelihood inferences on semiparametric varying-coefficient partially linear models. Bernoulli 11:1031–1057MathSciNetCrossRefzbMATHGoogle Scholar
  11. Gasser T, Sroka L, Jennen-Steinmetz C (1986) Residual variance and residual pattern in nonlinear regression. Biometrika 73:625–633MathSciNetCrossRefzbMATHGoogle Scholar
  12. Hall P, Kay JW, Titterington DM (1990) Asymptotically optimal difference-based estimation of variance in nonparametric regression. Biometrika 77:521–528MathSciNetCrossRefzbMATHGoogle Scholar
  13. Hall P, Keilegom IV (2003) Using difference-based methods for inference in nonparametric regression with time series errors. J Roy Stat Soc B 65:443–456MathSciNetCrossRefzbMATHGoogle Scholar
  14. Hardle W, Liang H, Gao J (2000) Partially linear models. Physika Verlag, HeidelbergCrossRefzbMATHGoogle Scholar
  15. He H, Tang W, Zuo G (2014) Statistical inference in the partial inear models with the doubles moothing local linear regression method. J Stat Plan Inference 146:102–112CrossRefGoogle Scholar
  16. Hu H, Zhang Y, Pan X (2016) Asymptotic normality of dhd estimators in a partially linear model. Stat Pap 57:567–587MathSciNetCrossRefzbMATHGoogle Scholar
  17. Levine M (2015) Minimax rate of convergence for an estimator of the functional component in a semiparametric multivariate partially linear model. J Multivar Anal 140:283–290MathSciNetCrossRefzbMATHGoogle Scholar
  18. Liu Q, Zhao G (2012) A comparison of estimation methods for partially linear models. Int J Innov Manag Inf Prod 3:38–42Google Scholar
  19. Lokshin M (2006) Difference-based semiparametric estimation of partial linear regression models. Stata J 6:377–383Google Scholar
  20. Rice JA (1984) Bandwidth choice for nonparametric regression. Ann Stat 12:1215–1230MathSciNetCrossRefzbMATHGoogle Scholar
  21. Schott JR (1997) Matrix analysis for statistics. Wiley, New YorkzbMATHGoogle Scholar
  22. Severini T, Wong W (1992) Generalized profile likelihhood and conditional parametric models. Ann Stat 20:1768–1802CrossRefzbMATHGoogle Scholar
  23. Spechman P (1988) Kernel smoothing in partial linear models. J Roy Stat Soc B 50:413–436MathSciNetGoogle Scholar
  24. Tabakan G (2013) Performance of the difference-based estimators in partially linear models. Statistics 47:329–347MathSciNetCrossRefzbMATHGoogle Scholar
  25. Tong T, Ma Y, Wang Y (2013) Optimal variance estimation without estimating the meam function. Bernoulli 19:1839–1854MathSciNetCrossRefzbMATHGoogle Scholar
  26. Tong T, Wang Y (2005) Estimating residual variance in nonparametric regression using least squares. Biometrika 92:821–830MathSciNetCrossRefzbMATHGoogle Scholar
  27. Wang L, Brown L, Cai T (2011) A difference based approach to the semiparametric partial linear model. Electron J Stat 5:619–641MathSciNetCrossRefzbMATHGoogle Scholar
  28. Wang W, Lin L (2015) Derivative estimation based on difference sequence via locally weighted least squares regession. J Mach Learn Res 16:2617–2641MathSciNetzbMATHGoogle Scholar
  29. Wang W, Lin L, Yu L (2017) Optimal variance estimation based on lagged second-order difference in nonparametric regression. Comput Stat 32:1047–1063MathSciNetCrossRefzbMATHGoogle Scholar
  30. Whittle P (1964) On the convergence to normality of quadratic forms in independent variables. Teor. Verojatnost. i Primenen 9:113–118MathSciNetzbMATHGoogle Scholar
  31. Yatchew A (1997) An elementary estimator of the partial linear model. Econ Lett 57:135–143MathSciNetCrossRefzbMATHGoogle Scholar
  32. Zhao H, You J (2011) Difference based estimation for partially linear regression models with measurement errors. J Multivar Anal 102:1321–1338MathSciNetCrossRefzbMATHGoogle Scholar
  33. Zhou Y, Cheng Y, Wang L, Tong T (2015) Optimal difference-based variance estimation in heteroscedastic nonparametric regression. Stat Sin 25:1377–1397MathSciNetzbMATHGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2017

Authors and Affiliations

  1. 1.School of Mathematics and Big DataAnhui University of Science and TechnologyHuainanChina
  2. 2.School of Statistics and MathematicsZhejiang Gongshang UniversityHangzhouChina
  3. 3.Glorious Sun School of Business and ManagementDonghua UniversityShanghaiChina
  4. 4.CEMSE DivisionKing Abdullah University of Science and TechnologyThuwalSaudi Arabia
  5. 5.Department of MathematicsHong Kong Baptist UniversityKowloonChina

Personalised recommendations