Abstract
The least absolute deviation (LAD) estimator is an alternative to the ordinary least squares estimator when some outliers exist, or the error term in the regression model has a heavy-tailed distribution. The gist of this chapter is to present a new estimator for sparse and robust linear regression that improves the preliminary test LAD estimator, an estimator which depends on a test decision. Our strategy is to apply auxiliary information in the estimation obtained from employing the LAD-LASSO operator to find the null hypothesis, building the preliminary test estimator and its improvement. A Monte-Carlo simulation study shows that this new estimator is better than others. Moreover, an objective data analysis confirms that our proposed estimator performs better in the prediction error sense than the LAD, LAD-LASSO, and preliminary test estimators.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Alfons, A., Croux, C., & Gelper, S. (2013). Sparse least trimmed squares regression for analyzing high-dimensional large data sets. The Annals of Applied Statistics, 7(1), 226–248.
Bancroft, T. A. (1944). On biases in estimating due to use of preliminary tests of significance. The Annals of Mathematical Statistics, 15(2), 195–204.
Bancroft, T. A. (1964). Analysis and inference for incompletely specified models involving the use of preliminary test(s) of significance. Biometrics, 20(3), 427–442.
Bancroft, T. A. (1965). Inference for incompletely specified models in the physical sciences (with discussion). Bulletin ISI, Proceedings 35th Session, 41(1), 497–515.
Barro, R., & Lee, J. W. (1994). Dataset for a panel of 138 countries, http://admin.nber.org/pub/barro.lee.
Chen, K., Ying, Z., Zhang, H., & Zhao, L. (2008). Analysis of least absolute deviation. Biometrika, 95(1), 107–122.
Fan, J., & Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association, 96, 1348–1360.
Filzmoser, P., Serneels, S., Maronna, R., & Croux, C. (2009). Robust multivariate methods in chemometrics. In Brown, S. D., Tauler, R., & Walczak, B. (Eds.), Comprehensive chemometrics: Chemical and biochemical data analysis (pp. 663–722). Elsevier. http://hdl.handle.net/20.500.12708/26445.
Hoerl, A. E., & Kennard, R. W. (1970). Ridge regression: Biased estimation for nonorthogonal problems. Technometrics, 12, 186–199.
Hoffmann, I., Serneels, S., Filzmoser, P., & Croux, C. (2015). Sparse partial robust M regression. Chemometrics and Intelligent Laboratory Systems, 149, 50–59.
Knight, K., & Fu, W. (2000). Asymptotics for lasso-type estimators. The Annals of Statistics, 28(5), 1356–1378.
Koenker, R. (2015). quantreg: Quantile regression, R package version 5.19.
Koenker, R., & Machado, J. A. F. (1999). Goodness of fit and related inference process for quantile regression. Journal of the American Statistical Association, 94, 1296–1310.
Kurnaz, F. S., Hoffmann, I., & Filzmoser, P. (2018). Robust and sparse estimation methods for high-dimensional linear and logistic regression. Chemometrics and Intelligent Laboratory Systems, 172, 211–222.
Norouzirad, M., & Arashi, M. (2019). Preliminary test and Stein-type shrinkage ridge estimators in robust regression. Statistical Papers, 60, 1849–1882.
Norouzirad, M., Hossain, S., & Arashi, M. (2018). Shrinkage and penalized estimators in weighted least absolute deviations regression models. Journal of Statistical Computation and Simulation, 88(8), 1557–1575.
Pollard, D. (1991). Asymptotic for least absolute deviation regression estimators. Econometric Theory, 7(2), 186–199.
Rahman, M., & Gokhale, D. V. (1996). Testimation in regression parameter estimation. Biometrical Journal, 38(7), 809–817.
Rosset, S., & Zhu, J. (2004). Discussion of “Least angle regression’’ by Efron et al. The Annals of Statistics, 32(2), 469–475.
Saleh, A. K. Md. E. (2006). Theory of preliminary test and stein-type estimation with applications. New York: Wiley.
Saleh, A. KMd. E., Arashi, M., & Tabatabaey, S. M. M. (2014). Statistical inference for models with multivariate t-distributed errors. New Jersy: Wiley.
Sclove, S. L., Morris, C., & Radhakrishnan, R. (1972). Non-optimality of preliminary-test estimators for the mean of a multivariate normal distribution. Annals of Mathematical Statistics, 43, 1481–1490.
Tibshirani, R. (1996). Regression shrinkage and selection via the LASSO. Journal of the Royal Statistical Society B, 58, 267–288.
Wang, H., Li. G., & Jiang, G. (2007). Robust regression shrinkage and consistent variable selection through the LAD-LASSO. Journal of Business & Economic Statistics, 25(3), 347–355.
Zhao, P., & Yu, B. (2006). On model selection consistency of LASSO. Journal of Machine Learning Research, 7, 2541–2563.
Acknowledgements
M. Norouzirad and F. J. Marques wish to acknowledge funding provided by the National Funds through the FCT—Fundação para a Ciência e a Tecnologia, I.P., under the scope of the projects UIDB/00297/2020 and UIDP/00297/2020 (Center for Mathematics and Applications).
M. Arashi’s work was based upon research supported in part by the National Research Foundation (NRF) of South Africa, SARChI Research Chair UID: 71199; Ref.: IFR170227223754 grant No. 109214. The opinions expressed and conclusions arrived at are those of the authors and are not necessarily attributed to the NRF.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Norouzirad, M., Arashi, M., Marques, F.J., Esmaeili, F. (2022). Robust Estimation Through Preliminary Testing Based on the LAD-LASSO. In: Bekker, A., Ferreira, J.T., Arashi, M., Chen, DG. (eds) Innovations in Multivariate Statistical Modeling. Emerging Topics in Statistics and Biostatistics . Springer, Cham. https://doi.org/10.1007/978-3-031-13971-0_19
Download citation
DOI: https://doi.org/10.1007/978-3-031-13971-0_19
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-13970-3
Online ISBN: 978-3-031-13971-0
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)