Exact finite-sample bias and MSE reduction in a simple linear regression model with measurement error

Abstract

This paper deals with the problem of estimating a slope parameter in a simple linear regression model, where independent variables have functional measurement errors. Measurement errors in independent variables, as is well known, cause biasedness of the ordinary least squares estimator. A general procedure for the bias reduction is presented in a finite sample situation, and some exact bias-reduced estimators are proposed. Also, it is shown that certain truncation procedures improve the mean square errors of the ordinary least squares and the bias-reduced estimators.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2

References

  1. Adcock, R. J. (1877). Note on the method of least squares. Analyst, 4, 183–184.

    Article  Google Scholar 

  2. Adcock, R. J. (1878). A problem in least squares. Analyst, 5, 53–54.

    Article  Google Scholar 

  3. Anderson, T. W. (1976). Estimation of linear functional relationships: Approximate distributions and connections with simultaneous equations in econometrics. Journal of the Royal Statistical Society Series B, 38, 1–36.

    MathSciNet  MATH  Google Scholar 

  4. Anderson, T. W. (1984). Estimating linear statistical relationships. The Annals of Statistics, 12, 1–45.

    MathSciNet  Article  Google Scholar 

  5. Aoki, M. (1989). Optimization of stochastic systems (2nd ed.). New York: Academic Press.

    Google Scholar 

  6. Berger, J. O., Berliner, L. M., & Zaman, A. (1982). General admissibility and inadmissibility results for estimation in a control problem. The Annals of Statistics, 10, 838–856.

    MathSciNet  Article  Google Scholar 

  7. Bickel, P. J., & Ritov, Y. (1987). Efficient estimation in the errors in variables model. The Annals of Statistics, 15, 513–540.

    MathSciNet  Article  Google Scholar 

  8. Brown, P. J. (1993). Measurement, regression, and calibration. Oxford: Oxford University Press.

    Google Scholar 

  9. Cheng, C.-L., & Van Ness, J. W. (1999). Statistical regression with measurement error. New York: Oxford University Press.

    Google Scholar 

  10. DeGracie, J. S., & Fuller, W. A. (1972). Estimation of the slope and analysis of covariance when the concomitant variable is measured with error. Journal of the American Statistical Association, 67, 930–937.

    MathSciNet  Article  Google Scholar 

  11. Fuller, W. A. (1987). Measurement error models. New York: Wiley.

    Google Scholar 

  12. Gleser, L. J. (1981). Estimation in a multivariate “errors in variables” regression model: Large sample results. The Annals of Statistics, 9, 24–44.

    MathSciNet  Article  Google Scholar 

  13. Guo, M., & Ghosh, M. (2012). Mean squared error of James-Stein estimators for measurement error models. Statistics & Probability Letters, 82, 2033–2043.

    MathSciNet  Article  Google Scholar 

  14. Huber, P. J. (1981). Robust Statistics. New York: Wiley.

    Google Scholar 

  15. Hudson, H. M. (1978). A natural identity for exponential families with applications in multiparameter estimation. The Annals of Statistics, 6, 473–484.

    MathSciNet  Article  Google Scholar 

  16. James, W., & Stein, C. (1961). Estimation with quadratic loss. In: Proc. Fourth Berkeley Symp. Math. Statist. Probab., 1, (pp. 361–379). Berkeley: Univ. California Press.

  17. Kendall, M. G., & Stuart, A. (1979). The advanced theory of statistics, (4th ed., Vol. 2). London: Griffin.

  18. Kubokawa, T. (1994). A unified approach to improving equivariant estimators. The Annals of Statistics, 22, 290–299.

    MathSciNet  Article  Google Scholar 

  19. Kubokawa, T., & Robert, C. P. (1994). New perspectives on linear calibration. Journal of Multivariate Analysis, 51, 178–200.

    MathSciNet  Article  Google Scholar 

  20. Nishii, R., & Krishnaiah, P. R. (1988). On the moments of classical estimates of explanatory variables under a multivariate calibration model. Sankhyā Series A, 50, 137–148.

    MathSciNet  MATH  Google Scholar 

  21. Osborne, C. (1991). Statistical calibration: A review. International Statistical Review, 59, 309–336.

    Article  Google Scholar 

  22. Reiersøl, W. (1950). Identifiability of linear relationship between variables are subject to error. Econometrica, 23, 375–389.

    MathSciNet  Article  Google Scholar 

  23. Stefanski, L. A. (1985). The effects of measurement error on parameter estimation. Biometrika, 72, 583–592.

    MathSciNet  Article  Google Scholar 

  24. Sundberg, R. (1999). Multivariate calibration—Direct and indirect regression methodology (with discussion). Scandinavian Journal of Statistics, 26, 161–207.

    MathSciNet  Article  Google Scholar 

  25. Tsukuma, H. (2018). Estimation in a simple linear regression model with measurement error. arXiv:1804.03029.

  26. Whittemore, A. S. (1989). Errors-in-variables regression using Stein estimates. Journal of the American Statistical Association, 43, 226–228.

    Google Scholar 

  27. Zaman, A. (1981). A complete class theorem for the control problem and further results on admissibility and inadmissibility. The Annals of Statistics, 9, 812–821.

    MathSciNet  Article  Google Scholar 

  28. Zellner, A. (1971). An introduction to Bayesian inference in econometrics. New York: Wiley.

    Google Scholar 

Download references

Acknowledgements

The author would like to thank the two reviewers for their careful review and for helpful comments and suggestions. This work was supported by Grant-in-Aid for Scientific Research (18K11201) from Japan Society for the Promotion of Science.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Hisayuki Tsukuma.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Tsukuma, H. Exact finite-sample bias and MSE reduction in a simple linear regression model with measurement error. Jpn J Stat Data Sci 2, 1–29 (2019). https://doi.org/10.1007/s42081-018-0025-3

Download citation

Keywords

  • Bias correction
  • Errors-in-variables model
  • Functional relationship
  • Mean square error
  • Multivariate calibration problem
  • Repeated measurement
  • Shrinkage estimator
  • Statistical control problem
  • Statistical decision theory
  • Structural relationship

Mathematics Subject Classifications

  • Primary 62F10
  • Secondary 62J07