Skip to main content
SpringerLink
Log in

Empirical Bayes estimation in a multiple linear regression model

  • Published:
Annals of the Institute of Statistical Mathematics Aims and scope Submit manuscript

Summary

Estimation of the vector β of the regression coefficients in a multiple linear regressionY=Xβ+ε is considered when β has a completely unknown and unspecified distribution and the error-vector ε has a multivariate standard normal distribution. The optimal estimator for β, which minimizes the overall mean squared error, cannot be constructed for use in practice. UsingX, Y and the information contained in the observation-vectors obtained fromn independent past experiences of the problem, (empirical Bayes) estimators for β are exhibited. These estimators are compared with the optimal estimator and are shown to be asymptotically optimal. Estimators asymptotically optimal with rates nearO(n −1) are constructed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Bennett, Kemble G. and Martz, H. F. (1972). A continuous empirical Bayes smoothing technique,Biometrika,59, 361–368.

    Article  MathSciNet  Google Scholar 

  2. Clemmer, B. A. and Krutchkoff, Richard G. (1968). The use of empirical Bayes estimators in a linear regression model,Biometrika,55, 525–534.

    Article  Google Scholar 

  3. Cogburn, R. (1965) On the estimation of a multivariate location parameter with squared error loss,Bernoulli (1723), Bayes (1763) and Laplace (1813) Anniversary Volume (eds. J. Neyman and L. Lecam), Springer-Verlag, Berlin, 24–29.

    Google Scholar 

  4. Efron, B. and Morris, C. (1972). Limiting the risk of Bayes and empirical Bayes estimators—Part II: The empirical Bayes case,J. Amer. Statist. Ass.,67, 130–139.

    MATH  Google Scholar 

  5. Griffin, Barry S. and Krutchkoff, Richard G. (1971). Optimal linear estimators: an empirical Bayes version with application to the binomial distribution,Biometrika,58, 195–201.

    Article  Google Scholar 

  6. James, W. and Stein, C. (1961). Estimation with quadratic loss,Proc. Fourth Berkeley Symp. Math. Statist. Prob.,1, Univ. California Press, 361–379.

    MathSciNet  MATH  Google Scholar 

  7. Johns, M. V., Jr. (1957). Nonparametric empirical Bayes procedures,Ann. Math. Statist.,28, 649–669.

    Article  MathSciNet  Google Scholar 

  8. Johns, M. V., Jr. and Van Ryzin, J. R. (1971). Convergence rates for empirical Bayes two-action problems I: Discrete case,Ann. Math. Statist. 42, 1521–1539.

    Article  MathSciNet  Google Scholar 

  9. Johns, M. V., Jr and Van Ryzin, J. R. (1972). Convergence rates for empirical Bayes two-action problems II: Continuous case,Ann. Math. Statist.,43, 934–947.

    Article  MathSciNet  Google Scholar 

  10. Kantor, M. (1967). Estimating the mean of a multivariate normal distribution with applications to time series and empirical Bayes estimation, Ph. D. dissertation Columbia Univ.

  11. Kendall, Maurice and Stuart, Alan. (1969).The Advanced Theory of Statistics, 3rd ed. Vol. I, Hafner Publ. Co., New York.

    MATH  Google Scholar 

  12. Lehmann, E. L. (1959).Testing of Statistical Hypothesis, Wiley, New York.

    MATH  Google Scholar 

  13. Lin, P. E. (1972). Rates of convergence in empirical Bayes estimation problems: Discrete case,Ann. Inst. Statist. Math.,24, 319–325.

    Article  MathSciNet  Google Scholar 

  14. Lin, P. E. (1975). Rates of convergence in empirical Bayes estimation problems: Continuous case,Ann. Statist.,3, 155–164.

    Article  MathSciNet  Google Scholar 

  15. Loève, Michel (1963).Probability Theory, 3rd ed. Van Nostrand, Princeton.

    MATH  Google Scholar 

  16. Maritz, J. S. (1969). Empirical Bayes estimation for continuous distributions,Biometrika,56, 349–359.

    Article  MathSciNet  Google Scholar 

  17. Maritz, J. S. and Lwin, T. (1975). Construction of simple empirical Bayes estimators,J. R. Satis. Soc., B,75, 421–425.

    MathSciNet  MATH  Google Scholar 

  18. Martz, H. and Krutchkoff, R. (1969). Empirical Bayes estimators in a multiple linear regression model,Biometrika,56, 367–374.

    Article  Google Scholar 

  19. Neyman, J. (1962). Two breakthroughs in the theory of statistical decision making,Rev. Int. Statist. Inst.,30, 11–27.

    Article  MathSciNet  Google Scholar 

  20. O'Bryan, Thomas E. and Susarla, V. (1976). Rates in the empirical Bayes estimation problem with nonidentical components: Case of normal distributions,Ann Inst. Statist. Math.,28, 389–397.

    Article  MathSciNet  Google Scholar 

  21. Rao, C. Radhakrishna (1975). Simultaneous estimation of parameters in different linear models and applications to biometric problems,Biometrika,31, 545–554.

    Article  MathSciNet  Google Scholar 

  22. Robbins, Herbert (1963). An empirical Bayes approach to statistics,Proc. 3rd Berkeley Symp. Math. Statist. Prob.,1, 157–163, University California Press.

    Google Scholar 

  23. Robbins, Herbert (1963). The empirical Bayes approach to the testing of statistical hypothesis,Rev. Int. Statist. Inst.,31, 195–208.

    Article  Google Scholar 

  24. Robbins, Herbert (1964). The empirical Bayes approach to statistical decision problems,Ann. Math. Statist.,35, 1–20.

    Article  MathSciNet  Google Scholar 

  25. Samuel, E. (1963). An empirical Bayes approach to the testing of certain parametric hypotheses,Ann. Math. Statist.,34, 1370–1385.

    Article  MathSciNet  Google Scholar 

  26. Singh, R. S. (1974). Estimation of derivatives of average of μ-densities and sequence-compound estmation in exponential families, RM-318, Dept. Statist. Prob., Michigan State University.

  27. Singh, R. S. (1976). Empirical Bayes estimation with convergence rates in non-continuous Lebesgue-exponential families,Ann. Statist.,4, 431–439.

    Article  MathSciNet  Google Scholar 

  28. Singh, R. S. (1977). Applications of estimators of a density and its derivatives to certain statistical problems,J. R. Statist. Soc., B,39, 357–363.

    MathSciNet  MATH  Google Scholar 

  29. Singh, R. S. (1979). Empirical Bayes estimation in Lebesgue-exponential families with rates near the best possible rate,Ann. Statist.,7, 890–902.

    Article  MathSciNet  Google Scholar 

  30. Singh, R. S. (1981). Speed of convergence in nonparametric estimation of a multivariate μ-density and its mixed partial derivatives,J. Statist. Plann. Inf.,5, 287–298.

    Article  MathSciNet  Google Scholar 

  31. Stein, C. (1960). Multiple regression,Contributions to Probability and Statistics—Essays in Honor of Harold Hotelling, Stanford University Press, 424–443.

  32. Wind, S. (1972). Stein-James estimators of a multivariate location parameters,Ann. Math. Statist.,43, 340–343.

    Article  Google Scholar 

  33. Wind, S. (1973). An empirical Bayes approach to multiple linear regression,Ann. Statist.,1, 93–103.

    Article  MathSciNet  Google Scholar 

  34. Yu, Benito (1971). Rates of convergence in empirical Bayes two-action and estimation problems and in sequence-compound estimation problems, RM-279, Dept. Statist. Prob., Michigan State University.

Download references

Author information

Authors and Affiliations

  1. University of Guelph, Guelph, Canada

    R. S. Singh

Authors
  1. R. S. Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

Additional information

Supported in part by a Natural Sciences and Engineering Research Council of Canada grant.

About this article

Cite this article

Singh, R.S. Empirical Bayes estimation in a multiple linear regression model. Ann Inst Stat Math 37, 71–86 (1985). https://doi.org/10.1007/BF02481081

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF02481081

Key words and phrases

Search

Navigation