Sparse Point Estimation for Bayesian Regression via Simulated Annealing
In the context of variable selection in a regression model, the classical Lasso based optimization approach provides a sparse estimate with respect to regression coefficients but is unable to provide more information regarding the distribution of regression coefficients. Alternatively, using a Bayesian approach is more advantageous since it gives direct access to the distribution which is usually summarized by estimating the expectation (not sparse) and variance. Additionally, to support frequent application requirements, heuristics like thresholding are generally used to produce sparse estimates for variable selection purposes. In this paper, we provide a more principled approach for generating a sparse point estimate in a Bayesian framework. We extend an existing Bayesian framework for sparse regression to generate a MAP estimate by using simulated annealing. We then justify this extension by showing that this MAP estimate is also sparse in the regression coefficients. Experiments on real world applications like the splice site detection and diabetes progression demonstrate the usefulness of the extension.
Unable to display preview. Download preview PDF.
- 2.Caron, F., Doucet, A.: Sparse Bayesian nonparametric regression. In: ICML 2008, pp. 88–95. ACM (2008)Google Scholar
- 5.van Gerven, M., Cseke, B., Oostenveld, R., Heskes, T.: Bayesian source localization with the multivariate laplace prior. In: Advances in Neural Information Processing Systems 22, pp. 1901–1909 (2009)Google Scholar
- 7.Gramacy, R.B., Polson, N.G.: Simulation-based Regularized Logistic Regression. ArXiv e-prints (May 2010)Google Scholar
- 12.Raman, S., Fuchs, T., Wild, P., Dahl, E., Roth, V.: The Bayesian Group-Lasso for analyzing contingency tables. In: Proceedings of the 26th International Conference on Machine Learning, pp. 881–888 (June 2009)Google Scholar
- 19.Yuan, M., Lin, Y.: Model selection and estimation in regression with grouped variables. J. Roy. Stat. Soc. B, 49–67 (2006)Google Scholar