Statistics and Computing

, Volume 27, Issue 1, pp 103–114 | Cite as

Multiple-population shrinkage estimation via sliced inverse regression

  • Tao Wang
  • Xuerong Meggie Wen
  • Lixing ZhuEmail author


The problem of dimension reduction in multiple regressions is investigated in this paper, in which data are from several populations that share the same variables. Assuming that the set of relevant predictors is the same across the regressions, a joint estimation and selection method is proposed, aiming to preserve the common structure, while allowing for population-specific characteristics. The new approach is based upon the relationship between sliced inverse regression and multiple linear regression, and is achieved through the lasso shrinkage penalty. A fast alternating algorithm is developed to solve the corresponding optimization problem. The performance of the proposed method is illustrated through simulated and real data examples.


Joint sparsity Multiple regressions Sliced inverse regression Sufficient dimension reduction 


  1. Bernard-Michel, C., Gardes, L., Girard, S.: A note on sliced inverse regression with regularizations. Biometrics 64, 982–984 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  2. Bernard-Michel, C., Gardes, L., Girard, S.: Gaussian regularized sliced inverse regression. Stat. Comput. 19, 85–98 (2009)MathSciNetCrossRefGoogle Scholar
  3. Bondell, H.D., Li, L.: Shrinkage inverse regression estimation for model-free variable selection. J. R. Stat. Soc. Ser. B 71, 287–299 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  4. Chavent, M., Kuentz, V., Liquet, B., Saracco, J.: Sliced inverse regression for stratified population. Commun. Stat.-Theory Methods 40, 1–22 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  5. Chen, C.H., Li, K.C.: Can SIR be as popular as multiple linear regression? Statistica Sinica 8, 289–316 (1998)MathSciNetzbMATHGoogle Scholar
  6. Chiaromonte, F., Cook, R.D., Li, B.: Sufficient dimension reduction in regressions with categorical predictors. Ann. Stat. 30, 475–497 (2002)MathSciNetCrossRefzbMATHGoogle Scholar
  7. Cook, R.D.: Regression Graphics: Ideas for Studying Regressions Through Graphics. Wiley, New York (1998)CrossRefzbMATHGoogle Scholar
  8. Cook, R.D.: Testing predictor contributions in sufficient dimension reduction. Ann. Stat. 32, 1061–1092 (2004)MathSciNetzbMATHGoogle Scholar
  9. Cook, R.D., Forzani, L.: Likelihood-based sufficient dimension reduction. J. Am. Stat. Assoc. 104, 197–208 (2009)Google Scholar
  10. Cook, R.D., Ni, L.: Sufficient dimension reduction via inverse regression: a minimum discrepancy approach. J. Am. Stat. Assoc. 100, 410–428 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  11. Cook, R.D., Weisberg, S.: Discussion of “Sliced inverse regression for dimension reduction” by Ker-Chau Li. J. Am. Stat. Assoc. 86, 328–332 (1991)zbMATHGoogle Scholar
  12. Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning: Prediction, Inference and Data Mining. Springer, New York (2009)CrossRefzbMATHGoogle Scholar
  13. Lee, K., Li, B., Chiaromonte, F.: A general theory for nonlinear sufficient dimension reduction: formulation and estimation. Ann Stat 41, 221–249 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  14. Li, B., Zha, H., Chiaromonte, F.: Contour regression: a general approach to dimension reduction. Ann. Stat. 33, 1580–1616 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  15. Li, B., Kim, M., Altman, N.: On dimension folding of matrix- or array-valued statistical objects. Ann. Stat. 38, 1094–1121 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  16. Li, B., Wang, S.: On directional regression for dimension reduction. J. Am. Stat. Assoc. 102, 997–1008 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  17. Li, K.C.: Sliced inverse regression for dimension reduction. J. Am. Stat. Assoc. 86, 316–327 (1991)MathSciNetCrossRefzbMATHGoogle Scholar
  18. Li, L., Yin, X.: Sliced inverse regression with regularizations. Biometrics 64, 124–131 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  19. Lin, Y., Zhang, H.H.: Component selection and smoothing in multivariate nonparametric regression. Ann. Stat. 34, 2272–2297 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  20. Lounici, K., Pontil, M., Tsybakov, A.B., Van De Geer, S.: Taking advantage of sparsity in multi-task learning. arXiv preprint arXiv:0903.1468 (2009)
  21. Ni, L., Cook, R.D., Tsai, C.L.: A note on shrinkage sliced inverse regression. Biometrika 92, 242–247 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  22. Scrucca, L.: Class prediction and gene selection for DNA microarrays using regularized sliced inverse regression. Comput. Stat. Data Anal. 52, 438–451 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  23. Tibshirani, R.: Regression shrinkage and selection via the lasso. J. R. Stat. Soc. Ser. B 58, 267–288 (1996)MathSciNetzbMATHGoogle Scholar
  24. Wang, H., Leng, C.: Unified LASSO estimation by least squares approximation. J. Am. Stat. Assoc. 102, 1039–1048 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  25. Wang, T., Zhu, L.X.: Sparse sufficient dimension reduction using optimal scoring. Comput. Stat. Data Anal. 57, 223–232 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  26. Weisberg, S.: Applied Linear Regression. Wiley, New York (2005)CrossRefzbMATHGoogle Scholar
  27. Wu, Y., Li, L.: Asymptotic properties of sufficient dimension reduction with a diverging number of predictors. Statistica Sinica 31, 707–730 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  28. Xia, Y., Tong, H., Li, W.K., Zhu, L.X.: An adaptive estimation of dimension reduction space. J. R. Stat. Soc. Ser. B 64, 363–410 (2002)MathSciNetCrossRefzbMATHGoogle Scholar
  29. Yuan, M., Lin, Y.: Model selection and estimation in regression with grouped variables. J. R. Stat. Soc. Ser. B 68, 49–67 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  30. Zhu, L.P., Wang, T., Zhu, L.X., Ferré, L.: Sufficient dimension reduction through discretization-expectation estimation. Biometrika 97, 295–304 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  31. Zhu, L.P., Zhu, L.X., Feng, Z.H.: Dimension reduction in regressions through cumulative slicing estimation. J. Am. Stat. Assoc. 105, 1455–1466 (2010)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  1. 1.Department of BiostatisticsYale UniversityNew HavenUSA
  2. 2.Department of Mathematics and StatisticsMissouri University of Science and TechnologyRollaUSA
  3. 3.Department of MathematicsHong Kong Baptist UniversityKowloon TongHong Kong
  4. 4.School of StatisticsBeijing Normal UniversityBeijingChina

Personalised recommendations