Statistics and Computing

, Volume 13, Issue 1, pp 67–80 | Cite as

An alternative to model selection in ordinary regression

  • Nicholas T. Longford


The weaknesses of established model selection procedures based on hypothesis testing and similar criteria are discussed and an alternative based on synthetic (composite) estimation is proposed. It is developed for the problem of prediction in ordinary regression and its properties are explored by simulations for the simple regression. Extensions to a general setting are described and an example with multiple regression is analysed. Arguments are presented against using a selected model for any inferences.

hypothesis testing mean squared error model selection single-model based estimator synthetic estimator two-stage procedure 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Berger J.O. and Pericchi L.R. 1996. The intrinsic Bayes factor for linear models. In: Bernardo J.M., Berger J.O., Dawid A.P., and Smith A.F.M. (Eds.), Bayesian Statistics vol. 5, Oxford University Press, Oxford, pp. 25–44.Google Scholar
  2. Dempster A.P., Laird N.M., and Rubin D.B. 1977. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society Ser. B 39: 1–28.Google Scholar
  3. Draper D. 1995. Assessment and propagation of model uncertainty. Journal of the Royal Statistical Society Ser. B 57: 45–97.Google Scholar
  4. George E.I., and Foster D.P. 2000. Calibration and empirical Bayes variable selection. Biometrika 87: 731–747.Google Scholar
  5. George E.I. and McCulloch R.E. 1997. Approaches to Bayesian variable selection. Statistica Sinica 7: 339–373.Google Scholar
  6. Ghosh M. and Rao J.N.K. 1994. Small area estimation: An appraisal. Statistical Science 9: 55–93.Google Scholar
  7. Hand D.J., Daley F., Lunn A.D., McConway K.J., and Ostrowski E. 1994. A Handbook of Small Data Sets. Chapman and Hall, London, UK.Google Scholar
  8. Kass R.E. 1993. Bayes factors in practice. The Statistician 42: 551–560.Google Scholar
  9. Kass R.E. and Raftery A.E. 1995. Bayes factors. Journal of the American Statistical Association 90: 773–795.Google Scholar
  10. Longford N.T. 1999. Multivariate shrinkage estimation of small area means and proportions. Journal of the Royal Statistical Society Ser. A 162: 227–245.Google Scholar
  11. Longford N.T. 2001. Synthetic estimators with moderating influence. The carryover in crossover trials revisited. Statistics in Medicine 20: 3189–3203.Google Scholar
  12. O'Hagan A. 1995. Fractional Bayes factors for model comparisons. Journal of the Royal Statistical Society Ser. B 57: 99–138.Google Scholar
  13. Seber G.A.F. 1977. Linear Regression Analysis. Wiley and Sons, New York.Google Scholar

Copyright information

© Kluwer Academic Publishers 2003

Authors and Affiliations

  • Nicholas T. Longford
    • 1
  1. 1.De Montfort UniversityLeicesterUK

Personalised recommendations