Correlation Scaled Principal Component Regression

  • Krishna Kumar Singh
  • Amit Patel
  • Chiranjeevi Sadu
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 736)

Abstract

Multiple Regression is a form of model for prediction purposes. With large number of predictor variables, the multiple regression becomes complex. It may underfit on higher number of dimension (variables) reduction. Most of the regression techniques are either correlation based or principal components based. The correlation based method becomes ineffective if the data contains a large amount of multicollinearity, and the principal component approach also becomes ineffective if response variables depends on variables with lesser variance. In this paper, we propose a Correlation Scaled Principal Component Regression (CSPCR) method which constructs orthogonal predictor variables having scaled by corresponding correlation with the response variable. That is, the construction of such predictors is done by multiplying the predictors with corresponding correlation with the response variable and then PCR is applied on a varying number of principal components. It allows higher reduction in the number of predictors, compared to other standard methods like Principal Component Regression (PCR) and Least Squares Regression (LSR). The computational results show that it gives a higher coefficient of determination than PCR, and simple correlation based regression (CBR).

Keywords

Multiple regression Principal components Correlations Multicollinearity 

References

  1. 1.
    Armstrong, J.S.: Illusions in regression analysis. Int. J. Forecast. 28(3), 689 (2012).  https://doi.org/10.1016/j.ijforecast.2012.02.001 CrossRefGoogle Scholar
  2. 2.
    Fisher, R.A.: The goodness of fit of regression formulae, and the distribution of regression coefficients. J. Roy. Stat. Soc. 85(4), 597–612 (1922).  https://doi.org/10.2307/2341124. JSTOR 2341124CrossRefGoogle Scholar
  3. 3.
    Kutner, M.H., Nachtsheim, C.J., Neter, J.: Applied Linear Regression Models, 4th edn., p. 25. McGraw-Hill/Irwin, Boston (2004)Google Scholar
  4. 4.
    Filzmoser, P., Croux, C.: Dimension reduction of the explanatory variables in multiple linear regression. Pliska Stud. Math. Bulgar. 14, 59–70 (2003)MathSciNetGoogle Scholar
  5. 5.
    Martens, H., Naes, T.: Multivariate Calibration. Wiley, London (1993)MATHGoogle Scholar
  6. 6.
    Basilevsky, A.: Statistical Factor Analysis and Related Methods: Theory and Applications. Wiley & Sons, New York (1994)CrossRefMATHGoogle Scholar
  7. 7.
    Araújo, M.C.U., Saldanha, T.C.B., Galvão, R.K.H., Yoneyama, T., Chame, H.C., Visani, V.: The successive projections algorithm for variable selection in spectroscopic multicomponent analysis. Chemom. Intell. Lab. Syst. 57, 65–73 (2001)CrossRefGoogle Scholar
  8. 8.
    Ng, K.S.: A Simple Explanation of Partial Least Squares, Draft, 27 April 2013Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Krishna Kumar Singh
    • 1
  • Amit Patel
    • 1
  • Chiranjeevi Sadu
    • 1
  1. 1.RGUKT IIIT NuzvidKrishnaIndia

Personalised recommendations