Robust PCA for High-dimensional Data

  • M. Hubert
  • P. J. Rousseeuw
  • S. Verboven


Principal component analysis (PCA) is a well-known technique for dimension reduction. Classical PCA is based on the empirical mean and covariance matrix of the data, and hence is strongly affected by outlying observations. Therefore, there is a huge need for robust PCA. When the original number of variables is small enough, and in particular smaller than the number of observations, it is known that one can apply a robust estimator of multivariate location and scatter and compute the eigenvectors of the scatter matrix.

The other situation, where there are many variables (often even more variables than observations), has received less attention in the robustness literature. We will compare two robust methods for this situation. The first one is based on projection pursuit (Li and Chen, 1985; Rousseeuw and Croux, 1993; Croux and Ruiz-Gazen, 1996, 2000; Hubert et al., 2002). The second method is a new proposal, which combines the notion of outlyingness (Stahel, 1981; Donoho, 1982) with the FAST-MCD algorithm (Rousseeuw and Van Driessen, 1999). The performance and the robustness of these two methods are compared through a simulation study. We also illustrate the new method on a chemometrical data set.


Covariance Matrix Mean Square Error Projection Pursuit Affine Subspace Robust Principal Component Analysis 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. G. Boente and L. Orellana. A robust approach to common principal components. In L.T. Fernholz, S. Morgenthaler, and W. Stahel, editors, Statistics in Genetics and in Environmental Sciences,pages 117–146. Birkhauser, Basel, 2001.CrossRefGoogle Scholar
  2. C. Croux and G. Haesbroeck. Principal component analysis based on robust estimators of the covariance or correlation matrix: Influence functions and efficiencies. Biometrika, 87: 603–618, 2000.MathSciNetMATHCrossRefGoogle Scholar
  3. C. Croux and A. Ruiz-Gazen. A fast algorithm for robust principal components based on projection pursuit. In A. Prat, editor, Proceedings in Computational Statistics, pages 211–217. Physica, Heidelberg, 1996.Google Scholar
  4. C. Croux and A. Ruiz-Gazen. High breakdown estimators for principal components: The projection-pursuit approach revisited. Université libre de Bruxelles, 2000. Preprint.Google Scholar
  5. P.L. Davies. Asymptotic behavior of S-estimators of multivariate location and dispersion matrices. Ann. Statist., 15:1269–1292, 1987.MathSciNetMATHCrossRefGoogle Scholar
  6. D.L. Donoho. Breakdown properties of multivariate location estimators. PhD thesis, Qualifying paper, Harvard University, Boston, 1982.Google Scholar
  7. M. Hubert, P.J. Rousseeuw, and S. Verboven. A fast method for robust principal components with applications to chemometrics. Chemometrics and Intelligent Laboratory Systems, 60:101–111,2002.CrossRefGoogle Scholar
  8. G. Li and Z. Chen. Projection-pursuit approach to robust dispersion matrices and principal components: primary theory and Monte Carlo. J. Am. Statist. Assoc.,80:759–766, 1985.MATHCrossRefGoogle Scholar
  9. B.D. Marx and P.H.C. Eilers. Generalized linear regression on sampled signals and curves: A P-spline approach. Technometrics, 41:1–13, 1999.CrossRefGoogle Scholar
  10. B.G. Osborne, T. Fearn, A.R. Miller, and S. Douglas. Application of near infrared reflectance spectroscopy to the compositional analysis of biscuits and biscuit dough. J. of the Science of Food and Agriculture, 35:99–105, 1984.CrossRefGoogle Scholar
  11. P.J. Rousseeuw. Least median of squares regression. J. Am. Statist. Assoc., 79:871–880, 1984.MathSciNetMATHCrossRefGoogle Scholar
  12. P.J. Rousseeuw and C. Croux. Alternatives to the median absolute deviation. J. Am. Statist. Assoc., 88:1273–1283, 1993.MathSciNetMATHCrossRefGoogle Scholar
  13. P.J. Rousseeuw and A.M. Leroy. Robust regression and outlier detection. Wiley, New York, 1987.MATHCrossRefGoogle Scholar
  14. P.J. Rousseeuw and K. Van Driessen. A fast algorithm for the minimum covariance determinant estimator. Technometrics, 41(3):212–223, 1999.CrossRefGoogle Scholar
  15. P.J. Rousseeuw and B.C. Van Zomeren. Unmasking multivariate outliers and leverage points. J. Am. Statist. Assoc.,85:633–651, 1990.CrossRefGoogle Scholar
  16. W.A. Stahel. Robust estimation: Infinitesimal optimality and covariance matrix estimators. PhD thesis, ETH, ZUrich, 1981.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2003

Authors and Affiliations

  • M. Hubert
    • 1
  • P. J. Rousseeuw
    • 2
  • S. Verboven
    • 2
  1. 1.Department of MathematicsCatholic University of LeuvenBelgium
  2. 2.Department of Mathematics and Computer ScienceUniversity of Antwerp (UIA)Belgium

Personalised recommendations