Abstract
The effect of multicollinearity deems the estimation and interpretation of regression coefficients by the method of Ordinary Least Squares (OLS) as burdensome. Even if there are alternate techniques like Ridge Regression (RR) and Liu Regression (LR) for providing a model, the statistical inference is impaired. In this study, we present the Simultaneous Raise Regression (SRR) approach, which is based on QR decomposition and reduces multicollinearity while preserving statistical inference. In contrast to the classical raise approach, the proposed strategy attains the number of variables to be raised and their corresponding raise parameter with a single step. We have designed Sequential Variation Inflation Factor for this purpose (SVIF). The performance of SRR has been compared with OLS, Ridge Regression and Liu Regression through Predictive Root Mean Square Error (PRMSE), Predictive Mean Absolute Error (PMAE) and Standard Error of the coefficients. To validate the suggested approach, a simulation study and a real-life example are presented; both outcomes imply that our proposed method outperforms the competition.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Belsley, D.A., Kuh, E., Welsch, R.E.: The condition number. Regression diagnostics: Identifying influential data and sources of collinearity 100, 104 (1980)
Belsley, D.A.: A guide to using the collinearity diagnostics. Comput. Sci. Econ. Manag. 4(1), 33–50 (1991)
Chatterjee, S., Price, B.: Regression Diagnostics. New York (1991)
Cule, E., Moritz, S., Frankowski, D.: ridge: ridge regression with automatic selection of the penalty parameter. R Package Version 3, 3 (2022)
Druilhet, P., Mom, A.: Shrinkage structure in biased regression. J. Multivar. Anal. 99(2), 232–244 (2008)
Efron, B., Hastie, T., Johnstone, I., Tibshirani, R.: Least angle regression. Ann. Stat. 32(2), 407–499 (2004)
Frisch, R.: Statistical confluence analysis by means of complete regression systems, volume 5. Universitetets Økonomiske Instituut (1934)
Garcia, C.G., Pérez, J.G., Liria, J.S.: The raise method. An alternative procedure to estimate the parameters in presence of collinearity. Qual. Quant. 45(2), 403–423 (2011)
García, C.B., Salmerón, R., García, C., García, J.: Residualization: justification, properties and application. J. Appl. Stat. 47(11), 1990–2010 (2020)
Giacalone, M., Panarello, D., Mattera, R.: Multicollinearity in regression: an efficiency comparison between LP-norm and least squares estimators. Qual. Quant. 52(4), 1831–1859 (2018)
Gómez, R.S., Sánchez, A.R., García, C.G., Pérez, J.G., et al.: The VIF and MSE in raise regression. Mathematics 8(4), 1–28 (2020)
Halawa, A., El Bassiouni, M.: Tests of regression coefficients under ridge regression models. J. Stat. Comput. Simul. 65(1–4), 341–356 (2000)
Hoerl, A.E., Kennard, R.W.: Ridge regression: applications to nonorthogonal problems. Technometrics 12(1), 69–82 (1970)
Hoerl, A.E., Kennard, R.W.: Ridge regression: biased estimation for nonorthogonal problems. Technometrics 12(1), 55–67 (1970)
Imdad, M.U., Aslam, M.: liureg: Liu regression with Liu biasing parameters and statistics. R Package Version 1(1), 2 (2018)
Imdadullah, M., Aslam, M.: liureg: A comprehensive r package for the Liu estimation of linear regression model with collinear regressors. R J. 9(2), 232–247 (2017)
Kejian, L.: A new class of blased estimate in linear regression. Commun. Stat. Theory Methods 22(2), 393–402 (1993)
Kendall, M.G.: A Course in Multivariate Analysis: London. Charles Griffin & Co, London (1957)
Klein, L.R., Nakamura, M.: Singularity in the equation systems of econometrics: some aspects of the problem of multicollinearity. Int. Econ. Rev. 3(3), 274–299 (1962)
Kovács, P., Petres, T., Tóth, L.: A new measure of multicollinearity in linear regression models. Int. Stat. Rev. 73(3), 405–412 (2005)
Kutner, M.H., Nachtsheim, C.J., Neter, J., Wasserman, W.: Applied linear regression models, volume 4. McGraw-Hill/Irwin New York (2004)
Lin, F.-J.: Solving multicollinearity in the process of fitting regression model using the nested estimate procedure. Qual. Quant. 42(3), 417–426 (2008)
Liu, X.-Q.: Improved liu estimator in a linear regression model. J. Stat. Plann. Inference 141(1), 189–196 (2011)
Marquardt, D.W.: Comment: You should standardize the predictor variables in your regression models. J. Am. Stat. Assoc. 75(369), 87–91 (1980)
Marquardt, D.W., Snee, R.D.: Ridge regression in practice. Am. Stat. 29(1), 3–20 (1975)
Midi, H., Bagheri, A.: Robust multicollinearity diagnostic measure in collinear data set. In: Proceedings of the 4th International Conference on Applied Mathematics, Simulation, Modeling (pp. 138–142).: World Scientific and Engineering Academy and Society (WSEAS) (2010)
Midi, H., Bagheri, A.: Robust multicollinearity diagnostic measures based on minimum covariance determination approach. Econ. Comput. Econ. Cybern. Stud. Res. 47(4), 1–15 (2013)
Obite, C., Olewuezi, N., Ugwuanyim, G., Bartholomew, D.: Multicollinearity effect in regression analysis: a feed forward artificial neural network approach. Asian J. Probab. Stat. 6(1), 22–33 (2020)
Özkale, M.R., Kaçıranlar, S.: A prediction-oriented criterion for choosing the biasing parameter in Liu estimation. Commun. Stat. Theory Methods 36(10), 1889–1903 (2007)
Perez-Melo, S., Kibria, B.: On some test statistics for testing the regression coefficients in presence of multicollinearity: a simulation study. Stats 3(1), 40–55 (2020)
Salmeron, R., Garcia, C., Garcia, J., Lopez, M.D.: The raise estimator estimation, inference, and properties. Commun. Stat. Theory Methods 46(13), 6446–6462 (2017)
Salmerón, R., García, C., García, J.: Overcoming the inconsistences of the variance inflation factor: a redefined VIF and a test to detect statistical troubling multicollinearity. Econometrica (2020)
Saville, D.J., Wood, G.R.: Statistical Methods: The Geometric Approach. Springer, Berlin (2012)
Sharma, S.: Applied Multivariate Techniques. John Wiley & Sons Inc, Hoboken (1996)
Stewart, G.W.: Collinearity and least squares regression. Stat. Sci. 2(1), 68–84 (1987)
Stine, R.A.: Graphical interpretation of variance inflation factors. Am. Stat. 49(1), 53–56 (1995)
Strang, G.: Introduction to linear algebra. 5th edn Cambridge. United Kingdom: Wellesley-Cambridge Press (2016)
Venables, W.N., Ripley, B.D.: Modern Applied Statistics with S. New York: Springer, fourth edition (2002)
Watkins, D.S.: Fundamentals of matrix computations. Math. Comput. 59, 299 (1992)
Acknowledgements
The authors would like to thank Quality & Quantity’s editors and the anonymous reviewers who provided valuable comments and helped improve the quality of this paper.
Funding
The authors declare that there is no funding for this study.
Author information
Authors and Affiliations
Contributions
Authors contributed to all sections of the manuscript.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that there are no conflicts of interest regarding the publication of this paper.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Jacob, J., Varadharajan, R. Simultaneous raise regression: a novel approach to combating collinearity in linear regression models. Qual Quant 57, 4365–4386 (2023). https://doi.org/10.1007/s11135-022-01557-9
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11135-022-01557-9