Statistics and Computing

, Volume 25, Issue 2, pp 261–272 | Cite as

Partitioning predictors in multivariate regression models

  • Francesca MartellaEmail author
  • Donatella Vicari
  • Maurizio Vichi


A Multivariate Regression Model Based on the Optimal Partition of Predictors (MRBOP) useful in applications in the presence of strongly correlated predictors is presented. Such classes of predictors are synthesized by latent factors, which are obtained through an appropriate linear combination of the original variables and are forced to be weakly correlated. Specifically, the proposed model assumes that the latent factors are determined by subsets of predictors characterizing only one latent factor. MRBOP is formalized in a least squares framework optimizing a penalized quadratic objective function through an alternating least-squares (ALS) algorithm. The performance of the methodology is evaluated on simulated and real data sets.


Penalized regression model Partition of variables Least squares estimation Class-correlated variables Latent factors 



The authors are grateful to the editor and anonymous referees of Statistics and Computing for their valuable comments and suggestions which improved the clarity and the relevance of the first version.


  1. Abdi, H.: Partial least squares regression and projection on latent structure regression (PLS-regression). Wiley Interdiscip. Rev.: Comput. Stat. 2, 97–106 (2010) CrossRefGoogle Scholar
  2. Abraham, B., Merola, G.: Dimensionality reduction approach to multivariate prediction. Comput. Stat. Data Anal. 48(1), 5–16 (2005) CrossRefzbMATHMathSciNetGoogle Scholar
  3. Anderson, T.W.: Estimating linear restrictions on regression coefficients for multivariate distributions. Ann. Math. Stat. 22, 327–351 (1951) CrossRefzbMATHGoogle Scholar
  4. Bougeard, S., Hanafi, M., Qannari, E.M.: Multiblock latent root regression: application to epidemiological data. Comput. Stat. 22(2), 209–222 (2007) CrossRefzbMATHMathSciNetGoogle Scholar
  5. Bougeard, S., Hanafi, M., Qannari, E.M.: Continuum redundancy-PLS regression: a simple continuum approach. Comput. Stat. Data Anal. 52(7), 3686–3696 (2008) CrossRefzbMATHMathSciNetGoogle Scholar
  6. Chauvin, C., Buuvrel, I., Belceil, P.A., Orand, J.P., Guillemot, D., Sanders, P.: A pharmaco-epidemiological analysis of factors associated with antimicrobial consumption level in turkey broiler flocks. Comput. Stat. Data Anal. 36, 199–211 (2005) Google Scholar
  7. De Jong, S.: SIMPLS: an alternative approach to partial least squares regression. Chemom. Intell. Lab. Syst. 18, 251–263 (1993) CrossRefGoogle Scholar
  8. Escoufier, Y.: Le traitement des variables vectorielles. Biometrics 29, 751–760 (1973) CrossRefMathSciNetGoogle Scholar
  9. Frank, I.E., Friedman, J.: A statistical view of some chemometrics regression tools. Technometrics 35, 109–148 (1993) CrossRefzbMATHGoogle Scholar
  10. Hocking, R.R.: The analysis and selection of variables in linear regression. Biometrics 32, 1–49 (1976) CrossRefzbMATHMathSciNetGoogle Scholar
  11. Hoerl, A.E., Kennard, R.W.: Ridge regression: biased estimation for non-orthogonal problems. Technometrics 12, 55–67 (1970) CrossRefzbMATHGoogle Scholar
  12. Hotelling, H.: The most predictable criterion. J. Educ. Psychol. 25, 139–142 (1935) CrossRefGoogle Scholar
  13. Hubert, L., Arabie, P.: Comparing partitions. J. Classif. 198, 193–218 (1985) CrossRefGoogle Scholar
  14. Izenman, A.J.: Reduced-rank regression for the multivariate linear model. J. Multivar. Anal. 5, 248–262 (1975) CrossRefzbMATHMathSciNetGoogle Scholar
  15. Jolliffe, I.T.: A note on the use of principal components in regression. J. R. Stat. Soc., Ser. C, Appl. Stat. 31(3), 300–303 (1982) Google Scholar
  16. Krzanowski, W.J.: Principles of Multivariate Analysis: A User’s Perspective. Oxford University Press, London (2000) Google Scholar
  17. Rosipal, R., Krämer, N.: Overview and recent advances in partial least squares. In: Subspace, Latent Structure and Feature Selection, vol. 3940, pp. 34–51 (2006) CrossRefGoogle Scholar
  18. Stone, M.: Cross-validation choice and assessment of statistical predictions. J. R. Stat. Soc. B 36, 111–147 (1974) zbMATHGoogle Scholar
  19. Stone, M., Brooks, R.J.: Continuum regression: cross-validated sequentially constructed prediction embracing ordinary least squares, partial least squares and principal components regression. J. R. Stat. Soc. B 52(2), 237–269 (1990) zbMATHMathSciNetGoogle Scholar
  20. Tibshirani, R.: Regression shrinkage and selection via the lasso. J. R. Stat. Soc. B 58(1), 267–288 (1996) zbMATHMathSciNetGoogle Scholar
  21. Tutz, G., Ulbricht, J.: Penalized regression with correlation-based penalty. Stat. Comput. 19(1), 239–253 (2009) CrossRefMathSciNetGoogle Scholar
  22. Van Den Wollenberg, A.L.: Redundancy analysis an alternative for canonical correlation analysis. Psychometrika 42(2), 207–219 (1977) CrossRefzbMATHGoogle Scholar
  23. Waldro, L., Pintilie, M., Tsao, M.S., Shepherd, F.A., Huttenhower, C., Jurisica, I.: Optimized application of penalized regression methods to diverse genomic data. Bioinformatics 27(24), 3399–3406 (2011) CrossRefGoogle Scholar
  24. Witten, D.M., Tibshirani, R.: Covariance-regularized regression and classification for high dimensional problems. J. R. Stat. Soc., Ser. B, Stat. Methodol. 71, 615–636 (2009) CrossRefzbMATHMathSciNetGoogle Scholar
  25. Wold, H.: Estimation of principal components and related models by iterative least squares. In: Krishnaiaah, P.R. (ed.) Multivariate Analysis, pp. 391–420. Academic Press, New York (1966) Google Scholar
  26. Yuan, M., Ekici, A., Lu, Z., Monteiro, R.: Dimension reduction and coefficient estimation in multivariate linear regression. J. R. Stat. Soc., Ser. B, Stat. Methodol. 69, 329–346 (2007) CrossRefMathSciNetGoogle Scholar
  27. Yuan, M., Lin, Y.: Model selection and estimation in regression with grouped variables. J. R. Stat. Soc. B 68, 49–67 (2006) CrossRefzbMATHMathSciNetGoogle Scholar
  28. Zou, H., Hastie, T.: Regularization and variable selection via the elastic net. J. R. Stat. Soc. B 67, 301–320 (2005) CrossRefzbMATHMathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Francesca Martella
    • 1
    Email author
  • Donatella Vicari
    • 1
  • Maurizio Vichi
    • 1
  1. 1.Dipartimento di Scienze StatisticheSapienza Università di RomaRomeItaly

Personalised recommendations