Abstract
We consider a joint processing of n independent similar sparse regression problems. Each is based on a sample \((y_{i1}, x_{i1})\ldots, (y_{im},x_{im})\) of m i.i.d. observations from \(y_{i1}=x_{i1}^T\;\beta_i+\epsilon_{i1},y_{i1}\in \mathbb{R},x_{i1}\in \mathbb{R}^p,\ {\rm and}\ \epsilon_{i1} \sim N(0, \sigma^2)\), say. The dimension p is large enough so that the empirical risk minimizer is not feasible. We consider, from a Bayesian point of view, three possible extensions of the lasso. Each of the three estimators, the lassoes, the group lasso, and the RING lasso, utilizes different assumptions on the relation between the n vectors \(\beta_1, \ldots, \beta_n\).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bickel, P., Ritov, Y., Tsybakov, A.: Simultaneous analysis of Lasso and Dantzig selector. Ann. Statist. 37, 1705–1732 (2009)
Bochkina, N., Ritov, Y.: Sparse empirical Bayes analysis (2009). URL http://arxiv.org/abs/0911.5482
Brown, L., Greenshtein, E.: Nonparametric empirical Bayes and compound decision approaches to estimation of a high-dimensional vector of normal means. Ann. Statist. 37, 1685–1704 (2009)
Greenshtein, E., Park, J., Ritov, Y.: Estimating the mean of high valued observations in high dimensions. J. Stat. Theory Pract. 2, 407–418 (2008)
Greenshtein, E., Ritov, Y.: Persistency in high dimensional linear predictor-selection and the virtue of over-parametrization. Bernoulli 10, 971–988 (2004)
Greenshtein, E., Ritov, Y.: Asymptotic efficiency of simple decisions for the compound decision problem. In: J. Rojo (ed.) Optimality: The 3rd Lehmann Symposium, IMS Lecture-Notes Monograph series, vol. 1, pp. 266–275 (2009)
Lounici, K., Pontil, M., Tsybakov, A.B., van de Geer, S.: Taking advantage of sparsity in multi-task learning. In: Proceedings of COLT’09, pp. 73–82 (2009)
Robbins, H.: Asymptotically subminimax solutions of compound decision problems. In: Proceedings of the 2nd Berkeley Symposium on Mathematical Statistics and Probability, vol. 1, pp. 131–148 (1951)
Robbins, H.: An empirical Bayes approach to statistics. In: Proceedings of the 3rd Berkeley Symposium on Mathematical Statistics and Probability, vol. 1, pp. 157–163 (1956)
Tibshirani, R.: Regression shrinkage and selection via the Lasso. J. R. Stat. Soc. Ser. B Stat. Methodol. 58, 267–288 (1996)
Yuan, M., Lin, Y.: Model selection and estimation in regression with grouped variables. J. R. Stat. Soc. Ser. B Stat. Methodol. 68, 49–67 (2006)
Zhang, C.H.: Compound decision theory and empirical Bayes methods. Ann. Statist. 31, 379–390 (2003)
Zhang, C.H.: General empirical Bayes wavelet methods and exactly adaptive minimax estimation.Ann. Statist. 33, 54–100 (2005)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Bochkina, N., Ritov, Y. (2011). Bayesian Perspectives on Sparse Empirical Bayes Analysis (SEBA). In: Alquier, P., Gautier, E., Stoltz, G. (eds) Inverse Problems and High-Dimensional Estimation. Lecture Notes in Statistics(), vol 203. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-19989-9_5
Download citation
DOI: https://doi.org/10.1007/978-3-642-19989-9_5
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-19988-2
Online ISBN: 978-3-642-19989-9
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)