Advertisement

Statistics and Computing

, Volume 19, Issue 1, pp 85–98 | Cite as

Gaussian Regularized Sliced Inverse Regression

  • Caroline Bernard-Michel
  • Laurent Gardes
  • Stéphane Girard
Article

Abstract

Sliced Inverse Regression (SIR) is an effective method for dimension reduction in high-dimensional regression problems. The original method, however, requires the inversion of the predictors covariance matrix. In case of collinearity between these predictors or small sample sizes compared to the dimension, the inversion is not possible and a regularization technique has to be used. Our approach is based on a Fisher Lecture given by R.D. Cook where it is shown that SIR axes can be interpreted as solutions of an inverse regression problem. We propose to introduce a Gaussian prior distribution on the unknown parameters of the inverse regression problem in order to regularize their estimation. We show that some existing SIR regularizations can enter our framework, which permits a global understanding of these methods. Three new priors are proposed leading to new regularizations of the SIR method. A comparison on simulated data as well as an application to the estimation of Mars surface physical properties from hyperspectral images are provided.

Keywords

Inverse regression Regularization Sufficient dimension reduction 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Amato, U., Antoniadis, A., De Feiss, I.: Dimension reduction in functional regression with applications. Comput. Stat. Data Anal. 50, 2422–2446 (2006) zbMATHCrossRefGoogle Scholar
  2. Antoniadis, A., Grégoire, G., McKeague, I.W.: Bayesian estimation in single-index models. Stat. Sin. 14, 1147–1164 (2004) zbMATHGoogle Scholar
  3. Aragon, Y., Saracco, J.: Sliced inverse regression: an appraisal of small sample alternatives to slicing. Comput. Stat. 12, 109–130 (1997) MathSciNetGoogle Scholar
  4. Bernard-Michel, C., Gardes, L., Girard, S.: Estimation of Mars surface physical properties from hyperspectral images using Sliced Inverse Regression. Technical Report, INRIA. http://hal.inria.fr/inria-00187444 (2007)
  5. Bernard-Michel, C., Gardes, L., Girard, S.: A note on Sliced Inverse Regression with regularizations. Biometrics (2008, to appear) Google Scholar
  6. Chiaromonte, F., Martinelli, J.: Dimension reduction strategies for analyzing global gene expression data with a response. Math. Biosci. 176, 123–144 (2002) zbMATHCrossRefMathSciNetGoogle Scholar
  7. Cook, R.D., Weisberg, S.: Discussion of “Sliced Inverse Regression for dimension reduction” by K.C. Li. J. Am. Stat. Assoc. 86, 328–332 (1991) CrossRefGoogle Scholar
  8. Cook, R.D.: Regression Graphics. Ideas for Studying Regressions Through Graphics. Wiley Series in Probability and Statistics. Wiley, New York (1998) zbMATHGoogle Scholar
  9. Cook, R.D.: Testing predictor contributions in sufficient dimension reduction. Ann. Stat. 32, 1062–1092 (2004) zbMATHCrossRefGoogle Scholar
  10. Cook, R.D.: Fisher lecture: Dimension reduction in regression. Stat. Sci. 22(1), 1–26 (2007) CrossRefGoogle Scholar
  11. Cook, R.D., Ni, L.: Sufficient dimension reduction via inverse regression: A minimum discrepancy approach. J. Am. Stat. Assoc. 100, 410–428 (2005) zbMATHCrossRefMathSciNetGoogle Scholar
  12. Cristianini, N., Shawe-Taylor, J.: An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods. Cambridge University Press, Cambridge (2000) Google Scholar
  13. Douté, S., Schmitt, B., Langevin, Y., Bibring, J.-P., Altieri, F., Bellucci, G., Gondet, B., Poulet, F.: South pole of Mars: Nature and composition of the icy terrains from Mars Express OMEGA observations. Planet. Space Sci. 55(1–2), 113–133 (2007) CrossRefGoogle Scholar
  14. Draper, N.R., Smith, R.: Applied Regression Analysis, 3rd edn. Wiley, New York (1998) zbMATHGoogle Scholar
  15. Ferré, L.: Determining the dimension in Sliced Inverse Regression and related methods. J. Am. Stat. Assoc. 93, 132–140 (1998) zbMATHCrossRefGoogle Scholar
  16. Ferré, L., Yao, A.F.: Smoothed functional inverse regression. Stat. Sin. 15, 665–683 (2005) zbMATHGoogle Scholar
  17. Friedman, J.H.: Regularized discriminant analysis. J. Am. Stat. Assoc. 84, 165–175 (1989) CrossRefGoogle Scholar
  18. Gannoun, A., Saracco, J.: An asymptotic theory for SIRα method. Stat. Sin. 13, 297–310 (2003) zbMATHMathSciNetGoogle Scholar
  19. Heiberger, R.: Generation of random orthogonal matrices. J. R. Stat. Soc. Ser. C 27, 199–206 (1978) zbMATHGoogle Scholar
  20. Hsing, T., Carroll, R.J.: An asymptotic theory for Sliced Inverse Regression. Ann. Stat. 20(2), 1040–1061 (1992) zbMATHCrossRefMathSciNetGoogle Scholar
  21. Li, K.C.: Sliced Inverse Regression for dimension reduction. J. Am. Stat. Assoc. 86, 316–327 (1991) zbMATHCrossRefGoogle Scholar
  22. Li, L., Li, H.: Dimension reduction methods for micro-arrays with application to censored survival data. Bioinformatics 20(18), 3406–3412 (2004) CrossRefGoogle Scholar
  23. Li, L., Yin, X.: Sliced Inverse Regression with regularizations. Biometrics 64(1), 124–131 (2008) zbMATHCrossRefMathSciNetGoogle Scholar
  24. Saracco, J.: An asymptotic theory for Sliced Inverse Regression. Commun. Stat. Theory Methods 26, 2141–2171 (1997) zbMATHCrossRefMathSciNetGoogle Scholar
  25. Saracco, J.: Asymptotics for pooled marginal slicing estimator based on SIRα approach. J. Multivar. Anal. 96(1), 117–135 (2005) zbMATHCrossRefMathSciNetGoogle Scholar
  26. Schmidt, F., Douté, S., Schmitt, B.: Wavanglet: an efficient supervised classifier for hyperspectral images. Geosci. Remote Sens. IEEE Trans. 45(5), 1374–1385 (2007) CrossRefGoogle Scholar
  27. Schott, J.R.: Determining the dimensionality in Sliced Inverse Regression. J. Am. Stat. Assoc. 89, 141–148 (1994) zbMATHCrossRefMathSciNetGoogle Scholar
  28. Scrucca, L.: Regularized Sliced Inverse Regression with applications in classification. In: Zani, S., Cerioli, A., Riani, M., Vichi, M. (eds.) Data Analysis, Classification and the Forward Search, pp. 59–66. Springer, Berlin (2006) CrossRefGoogle Scholar
  29. Scrucca, L.: Class prediction and gene selection for DNA microarrays using regularized Sliced Inverse Regression. Comput. Stat. Data Anal. 52, 438–451 (2007) CrossRefMathSciNetGoogle Scholar
  30. Vogel, C.R.: Computational Methods for Inverse Problems. SIAM, Philadelphia (2002) zbMATHGoogle Scholar
  31. Zhong, W., Zeng, P., Ma, P., Liu, J.S., Zhu, Y.: RSIR: Regularized Sliced Inverse Regression for motif discovery. Bioinformatics 21(22), 4169–4175 (2005) CrossRefGoogle Scholar
  32. Zhu, L.X., Ng, K.W.: Asymptotics of Sliced Inverse Regression. Stat. Sin. 5, 727–736 (1995) zbMATHMathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2008

Authors and Affiliations

  • Caroline Bernard-Michel
    • 1
  • Laurent Gardes
    • 1
  • Stéphane Girard
    • 1
  1. 1.Laboratoire Jean-Kuntzmann & INRIA Rhône-Alpes, Team MistisSaint-Ismier cedexFrance

Personalised recommendations