An Efficient Model Averaging Procedure for Logistic Regression Models Using a Bayesian Estimator with Laplace Prior

Abstract

Modern statistics has developed numerous methods for linear and nonlinear regression models, but the correct treatment of model uncertainty is still a difficult task. One approach is model selection, where, usually in a stepwise procedure, an optimal model is searched with respect to some (asymptotic) criterion such as AIC or BIC. A draw back of this approach is, that the reported post model selection estimates, especially for the standard errors of the parameter estimates, are too optimistic. A second approach is model averaging, either frequentist (FMA) or Bayesian (BMA). Here, not an optimal model is searched for, but all possible models are combined by some weighting procedure. Although conceptually easy, the approach has mainly one drawback: the number of potential models can be so large that it is infeasible to calculate the estimates for every possible model. In our paper we extend an idea of Magnus et al. (2009), called WALS, to the case of logistic regression. In principal, the method is not restricted to logistic regression but can be applied to any generalized linear model. In the final stage it uses a Bayesian esimator using a Laplace prior with a special hyperparameter.