Investigation about a screening step in model selection
In many studies a large number of variables is measured and the identification of relevant variables influencing an outcome is an important task. For variable selection several procedures are available. However, focusing on one model only neglects that there usually exist other equally appropriate models. Bayesian or frequentist model averaging approaches have been proposed to improve the development of a predictor. With a larger number of variables (say more than ten variables) the resulting class of models can be very large. For Bayesian model averaging Occam’s window is a popular approach to reduce the model space. As this approach may not eliminate any variables, a variable screening step was proposed for a frequentist model averaging procedure. Based on the results of selected models in bootstrap samples, variables are eliminated before deriving a model averaging predictor. As a simple alternative screening procedure backward elimination can be used.
Through two examples and by means of simulation we investigate some properties of the screening step. In the simulation study we consider situations with fifteen and 25 variables, respectively, of which seven have an influence on the outcome. With the screening step most of the uninfluential variables will be eliminated, but also some variables with a weak effect. Variable screening leads to more applicable models without eliminating models, which are more strongly supported by the data. Furthermore, we give recommendations for important parameters of the screening step.
KeywordsModel selection uncertainty Variable screening Bootstrap Simulation
Unable to display preview. Download preview PDF.
- Akaike, H.: Information theory and an extension of the maximum likelihood principle. In: Petrov, B., Csaki, B. (eds.) Second International Symposium on Information Theory, pp. 267–281. Academiai Kiado, Budapest (1973) Google Scholar
- Buchholz, A., Sauerbrei, W., Holländer, N.: On properties of predictors derived with a two-step bootstrap model averaging approach—a simulation study in the linear regression model. Comput. Stat. Data Anal. (2007, in press) Google Scholar
- Holländer, N., Augustin, N.H., Sauerbrei, W.: Investigation on the improvement of prediction by bootstrap model averaging. Methods Inf. Med. 45, 44–50 (2006) Google Scholar
- Hosmer, D.W., Lemeshow, S.: Applied Logistic Regression. Wiley, New York (2001) Google Scholar
- Johnson, R.W.: Fitting percentage of body fat to simple body measurements. J. Stat. Educ. 4 (1996) Google Scholar
- Teräsvirta, T., Mellin, I.: Model selection criteria and model selection tests in regression models. Scand. J. Stat. 13, 159–171 (1986) Google Scholar
- Wyatt, J.C., Altman, D.G.: Prognostic models: clinically useful or quickly forgotten? Br. Med. J. 311, 1539–1541 (1995) Google Scholar