Abstract
One of the most commonly used methods to prevent overfitting and select relevant variables in regression models with many predictors is the penalized regression technique. Under such approaches, variable selection is performed in a non-probabilistic way, using some optimization criterion.
A Bayesian approach to penalized regression has been proposed by assuming a prior distribution for the regression coefficients that plays a similar role as the penalty term in classical statistics: to shrink non-significant coefficients toward zero and assign a significant probability mass to non-negligible coefficients.
These prior distributions, called shrinkage priors, usually assume independence among the covariates, which may not be an appropriate assumption in many cases. We propose two shrinkage priors to model the uncertainty about coefficients that are spatially correlated.
The proposed priors are considered as an alternative approach to model the uncertainty about the coefficients of categorical variables with many levels. To illustrate their use, we consider the linear regression model. We evaluate the proposed method through several simulation studies.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Criscuolo, T.L.: Modelo partição produto para atributos categóricos. Ph.D. thesis, Instituto de Ciências Exatas da Universidade Federal de Minas Gerais (2019)
Derksen, S., Keselman, H.J.: Backward, forward and stepwise automated subset selection algorithms: frequency of obtaining authentic and noise variables. Br. J. Math. Stat. Psychol. 45(2), 265–282 (1992)
George, E.I., McCulloch, R.E.: Variable selection via Gibbs sampling. J. Am. Stat. Assoc. 88(423), 881–889 (1993)
Gertheiss, J., Tutz, G.: Sparse modeling of categorial explanatory variables. Ann. Appl. Stat. 4(4), 2150–2180 (2010)
Hans, C.: Bayesian lasso regression. Biometrika 96(4), 835–845 (2009)
Hoerl, A.E., Kennard, R.W.: Ridge regression: biased estimation for nonorthogonal problems. Technometrics 12, 55–67 (1970)
James, G., Witten, D., Hastie, T., Tibshirani, R.: An Introduction to Statistical Learning: With Applications in R. Springer, New York (2013). https://doi.org/10.1007/978-1-4614-7138-7
Kuo, L., Mallick, B.: Variable selection for regression models. Indian J. Stat. Ser. B 60, 65–81 (1998)
Kyung, M., Gill, J., Ghosh, M., Casella, G.: Penalized regression, standard errors, and Bayesian lassos. Bayesian Anal. 5(2), 369–411 (2010)
Li, F., Zhang, N.R.: Bayesian variable selection in structured high-dimensional covariate spaces with applications in genomics. J. Am. Stat. Assoc. 105(491), 1202–1214 (2010)
Li, Q., Lin, N.: The Bayesian elastic net. Bayesian Anal. 5(1), 151–170 (2010)
Liu, F., Chakraborty, S., Li, F., Liu, Y., Lozano, A.C.: Bayesian regularization via graph Laplacian. Bayesian Anal. 9(2), 449–474 (2014)
Park, T., Casella, G.: BAMLSS: the Bayesian Lasso. J. Am. Stat. Assoc. 103(482), 681–685 (2008)
Pauger, D., Wagner, H.: Bayesian effect fusion for categorical predictors. Bayesian Anal., 341–369 (2017)
Smith, M., Kohn, R.: Nonparametric regression using Bayesian variable selection. J. Econom. 75(2), 317–343 (1996)
Tibshirani, R.: Regression shrinkage and selection via the lasso. J. Royal Stat. Soc. (Ser. B) 58, 267–288 (1996)
Van Erp, S., Oberski, D., Mulder, J.: Shrinkage priors for Bayesian penalized regression. J. Math. Psychol. 89, 31–50 (2019)
Vannucci, M., Stingo, F., Berzuini, C.: Bayesian Models for Variable Selection That Incorporate Biological Information. Oxford University Press (2012). 9780199694587. Publisher Copyright: \(\copyright \) Oxford University Press 2011. All rights reserved. Copyright: Copyright 2018 Elsevier B.V., All rights reserved
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Cruz-Reyes, D. (2024). Spatial Shrinkage Prior: A Probabilistic Approach to Model for Categorical Variables with Many Levels. In: Orjuela-Cañón, A.D., Lopez, J.A., Arias-Londoño, J.D. (eds) Applications of Computational Intelligence. ColCACI 2023. Communications in Computer and Information Science, vol 1865. Springer, Cham. https://doi.org/10.1007/978-3-031-48415-5_11
Download citation
DOI: https://doi.org/10.1007/978-3-031-48415-5_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-48414-8
Online ISBN: 978-3-031-48415-5
eBook Packages: Computer ScienceComputer Science (R0)