Abstract
This contribution gives a brief review on penalized least squares methods in sparse linear regression models with a specific focus on heteroscedastic data structures. We discuss the well known bridge estimators, Lasso and adaptive Lasso and a new class of weighted penalized least squares methods, which address the problem of heteroscedasticity. We give a careful explanation on how the choice of the regularizing parameter affects the quality of the statistical inference (such as conservative or consistent model selection). The new estimators are asymptotically (pointwise) as efficient as estimators which are assisted by a model selection oracle. The results are illustrated by means of a small simulation study and the analysis of a data example.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Breiman, L. (1995). Better subset regression using the nonnegative garrote. Technometrics, 37, 373–384.
Candes, E., & Tao, T. (2007). The Dantzig selector: statistical estimation when p is much larger than n. The Annals of Statistics, 35, 2313–2351.
Dette, H., & Munk, A. (1998). Testing heteroscedasticity in nonparametric regression. Journal of the Royal Statistical Society. Series B. Statistical Methodology, 60, 693–708.
Efron, B., Hastie, T., & Tibshirani, R. (2004). Least angle regression (with discussion). The Annals of Statistics, 32, 407–451.
Fan, J., & Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association, 96, 1348–1360.
Fan, J., & Peng, H. (2004). Nonconcave penalized likelihood with a diverging number of parameters. The Annals of Statistics, 32, 928–961.
Frank, I. E., & Friedman, J. H. (1993). A statistical view of some chemometrics regression tools (with discussion). Technometrics, 35, 109–148.
Huang, J., Horowitz, J. L., & Ma, S. (2008a). Asymptotic properties of bridge estimators in sparse high dimensional regression models. The Annals of Statistics, 36, 587–613.
Huang, J., Ma, S., & Zhang, C. (2008b). Adaptive lasso for sparse high-dimensional regression models. Statistica Sinica, 18, 1603–1618.
Kim, Y., Choi, H., & Oh, H.-S. (2008). Smoothly clipped absolute deviation on high dimensions. Journal of the American Statistical Association, 103, 1665–1673.
Knight, F., & Fu, W. (2000). Asymptotics for Lasso-type estimators. The Annals of Statistics, 28, 1356–1378.
Leeb, H., & Pötscher, B. M. (2005). Model selection and inference: facts and fiction. Econometric Theory, 21, 21–59.
R Development Core Team (2008). R: A language and environment for statistical computing. R Foundation for Statistical Computing. Vienna, Austria. ISBN 3-900051-07-0. http://www.R-project.org
Tibshirani, R. (1996). Regression shrinkage and selection via the Lasso. Journal of the Royal Statistical Society. Series B. Statistical Methodology, 58, 267–288.
Wagener, J., & Dette, H. (2012). Bridge estimators and the adaptive Lasso under heteroscedasticity. Mathematical Methods of Statistics. doi:10.3103/S1066530712020032
Zhao, P., & Yu, B. (2006). On model selection consistency of Lasso. Journal of Machine Learning Research, 7, 2541–2563.
Zou, H. (2006). The adaptive Lasso and its oracle properties. Journal of the American Statistical Association, 101, 1418–1429.
Zou, H., & Hastie, T. (2005). Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society. Series B. Statistical Methodology, 67, 301–320.
Acknowledgements
This work has been supported in part by the Collaborative Research Center “Statistical modeling of nonlinear dynamic processes” (SFB 823, Teilprojekt C1) of the German Research Foundation (DFG). The authors would also like to thank Torsten Hothorn for pointing out some important references on the subject.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Dette, H., Wagener, J. (2013). Least Squares Estimation in High Dimensional Sparse Heteroscedastic Models. In: Becker, C., Fried, R., Kuhnt, S. (eds) Robustness and Complex Data Structures. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-35494-6_9
Download citation
DOI: https://doi.org/10.1007/978-3-642-35494-6_9
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-35493-9
Online ISBN: 978-3-642-35494-6
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)