Abstract
In the objective Bayesian approach to variable selection in regression a crucial point is the encompassing of the underlying nonnested linear models. Once the models have been encompassed, one can define objective priors for the multiple testing problem involved in the variable selection problem.
There are two natural ways of encompassing: one way is to encompass all models into the model containing all possible regressors, and the other is to encompass the model containing only the intercept into any other.
In this paper we compare the variable selection procedures that result from each of the two mentioned ways of encompassing by analysing their theoretical properties and their behavior with simulated and real data.
Relations with frequentest criteria for model selection, such as those based on the R 2adj and Mallows C p , are provided incidentally.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Atkinson AC (1978) Posterior probabilities for choosing a regression model. Biometrika 65:39–48
Berger JO, Bernardo JM (1992) On the development of reference priors (with discussion). In: Bernardo JM, Berger JO Dawid AP, Smith AFM (eds) Bayesian statistics, vol 4. Oxford University Press, Oxford, pp 35–60
Berger JO, Pericchi LR (1996) The intrinsic Bayes factor for model selection and prediction. J Am Stat Assoc 91:109–122
Berger JO, Pericchi LR (2004) Training samples in objective Bayesian model selection. Ann Stat 32:841–869
Box GEP, Meyer RD (1986) An analysis for unreplicated fractional factorial. Technometrics 28:11–18
Casella G, Moreno E (2005) Intrinsic meta-analysis of contingency tables. Stat Med 24:583–604
Casella G, Moreno E (2006) Objective Bayesian variable selection. J Am Stat Assoc 101:157–167
Clyde M, De Simone H, Parmigiani G (1996) Prediction via orthogonalized model mixing. J Am Stat Assoc 91:1197–1208
George EI, McCulloch RE (1993) Variable selection via Gibbs sampling. J Am Stat Assoc 88:881–889
George EI, McCulloch RE (1995) Stochastic search variable selection. In: Gilks WR, Richardson S, Spegelhalter DJ (eds) Practical Markov chain Monte Carlo in practice. Chapman and Hall, London, pp 339–348
George EI, McCulloch RE (1997) Approaches for variable selection. Stat Sin 7:339–373
Geweke J (1996) Variable selection and model comparison in regression. In: Bernardo JM, Berger JO, Dawid AP, Smith AFM (eds) Bayesian statistics, vol 5. Oxford University Press, Oxford, pp 169–194
Girón FJ, Martínez ML, Moreno E, Torres F (2006a) Objective testing procedures in linear models. Calibration of the p-values. Scand J Stat 33:765–784
Girón FJ, Moreno E, Martínez ML (2006b) An objective Bayesian procedure for variable selection in regression. In: Balakrishnan N, Castillo E, Sarabia JM (eds) Advances on distribution theory, order statistics and inference. Statistics for industry and technology. Birkhäuser, Boston, pp 393–408
Guttman I, Peña D, Redondas D (2005) A Bayesian approach for predicting with polynomial regression of unknown degree. Technometrics 47:23–33
Hastie T, Tibshirani R, Friedman J (2001) The elements of statistical learning: data mining, inference and prediction. Springer, New York
Kuo L, Mallick B (1998) Variable selection for regression models. Sankhyā 60:65–81
Mitchell TJ, Beauchamp JJ (1988) Bayesian variable selection in linear regression (with discussion). J Am Stat Assoc 83:1023–1036
Moreno E, Bertolino F, Racugno W (1998) An intrinsic limiting procedure for model selection and hypotheses testing. J Am Stat Assoc 93:1451–1460
Moreno E, Girón FJ (2005) Consistency of Bayes factors for intrinsic priors in normal linear models. Comptes Rendus Math Acad Des Sci Paris 340(11):911–914
Moreno E, Girón FJ, Torres F (2003) Intrinsic priors for hypothesis testing in normal regression models. RACSAM Revista R Acad Cienc Exactas Fís Nat Ser A Mat 97(1):53–61
Morris CM (1987) Discussion of Casella and Berger. J Am Stat Assoc 82:106–111
Pericchi LR (1984) An alternative to the standard Bayesian procedure for discrimination between normal linear models. Biometrika 71:575–586
Poirier DJ (1985) Bayesian hypothesis testing in linear models with continuously induced conjugate prior across hypotheses. In: Bernardo JM, DeGroot MH, Lindley DV, Smith AFM (eds) Bayesian statistics, vol 2. North-Holland, Amsterdam, pp 711–722
Smith AFM, Spiegelhalter DJ (1980) Bayes factor and choice criteria for linear models. J Roy Stat Soc Ser B 42:213–220
Smith M, Kohn R (1996) Nonparametric regression using Bayesian variable selection. J Econom 75:317–344
Snedecor J, Cochran W (1982) Statistical methods, 7th edn. Iowa State University Press
Spiegelhalter DJ, Smith AFM (1982) Bayes factor for linear and log–linear models with vague prior information. J Roy Stat Soc Ser B 44:377–387
Stamey T, Kabakin J, McNeal J, Johnstone I, Freiha F, Redwine E, Yang N (1989) Prostate specific antigen in the diagnosis and treatment of adenocarcinoma of the prostate II: radical prostatectomy treated patients. J Urol 16:1076–1083
Author information
Authors and Affiliations
Corresponding author
Additional information
This work has partially supported by grants SEJ 2004-02447 of MEC, Madrid, Spain and FQM-140.
An erratum to this article can be found at http://dx.doi.org/10.1007/s11749-008-0095-9
Rights and permissions
About this article
Cite this article
Moreno, E., Girón, F.J. Comparison of Bayesian objective procedures for variable selection in linear regression. TEST 17, 472–490 (2008). https://doi.org/10.1007/s11749-006-0039-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11749-006-0039-1