Principal Component Analysis

  • René Vidal
  • Yi Ma
  • S. Shankar Sastry
Part of the Interdisciplinary Applied Mathematics book series (IAM, volume 40)


Principal component analysis (PCA) is the problem of fitting a low-dimensional affine subspace to a set of data points in a high-dimensional space. PCA is, by now, well established in the literature, and has become one of the most useful tools for data modeling, compression, and visualization.


Asymptotic Mean Squared Error (AMSE) Probabilistic Principal Component Analysis (PPCA) PPCA Model Singular Value Thresholding Hard Thresholding 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. Akaike, H. (1977). A new look at the statistical model selection. IEEE Transactions on Automatic Control, 16(6), 716–723.MathSciNetzbMATHGoogle Scholar
  2. Basri, R., & Jacobs, D. (2003). Lambertian reflection and linear subspaces. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(2), 218–233.CrossRefGoogle Scholar
  3. Beltrami, E. (1873). Sulle funzioni bilineari. Giornale di Mathematiche di Battaglini, 11, 98–106.zbMATHGoogle Scholar
  4. Cai, J.-F., Candés, E. J., & Shen, Z. (2008). A singular value thresholding algorithm for matrix completion. SIAM Journal of Optimization, 20(4), 1956–1982.MathSciNetCrossRefzbMATHGoogle Scholar
  5. Cattell, R. B. (1966). The scree test for the number of factors. Multivariate Behavioral Research, 1, 245–276.CrossRefGoogle Scholar
  6. Collins, M., Dasgupta, S., & Schapire, R. (2001). A generalization of principal component analysis to the exponential family. In Neural Information Processing Systems (Vol. 14)Google Scholar
  7. Ding, C., Zha, H., He, X., Husbands, P., & Simon, H. D. (2004). Link analysis: Hubs and authoraties on the world wide web. SIAM Review, 46(2), 256–268.MathSciNetCrossRefzbMATHGoogle Scholar
  8. Donoho, D., & Gavish, M. (2014). The optimal hard threshold for singular values is \(4/\sqrt{3}\). IEEE Transactions on Information Theory, 60(8), 5040–5053.MathSciNetCrossRefGoogle Scholar
  9. Eckart, C., & Young, G. (1936). The approximation of one matrix by another of lower rank. Psychometrika, 1, 211–218.CrossRefzbMATHGoogle Scholar
  10. Gabriel, K. R. (1978). Least squares approximation of matrices by additive and multiplicative models. Journal of the Royal Statistical Society B, 40, 186–196.MathSciNetzbMATHGoogle Scholar
  11. Georghiades, A., Belhumeur, P., & Kriegman, D. (2001). From few to many: Illumination cone models for face recognition under variable lighting and pose. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(6), 643–660.CrossRefGoogle Scholar
  12. Hansen, M., & Yu, B. (2001). Model selection and the principle of minimum description length. Journal of American Statistical Association, 96, 746–774.MathSciNetCrossRefzbMATHGoogle Scholar
  13. Hotelling, H. (1933). Analysis of a complex of statistical variables into principal components. Journal of Educational Psychology, 24, 417–441.CrossRefzbMATHGoogle Scholar
  14. Householder, A. S., & Young, G. (1938). Matrix approximation and latent roots. American Mathematical Monthly, 45, 165–171.MathSciNetCrossRefzbMATHGoogle Scholar
  15. Hubert, L., Meulman, J., & Heiser, W. (2000). Two purposes for matrix factorization: A historical appraisal. SIAM Review, 42(1), 68–82.MathSciNetCrossRefzbMATHGoogle Scholar
  16. Jolliffe, I. (1986). Principal Component Analysis. New York: Springer.CrossRefzbMATHGoogle Scholar
  17. Jolliffe, I. (2002). Principal Component Analysis (2nd ed.). New York: Springer.zbMATHGoogle Scholar
  18. Jordan, M. (1874). Mémoire sur les formes bilinéaires. Journal de Mathématiques Pures et Appliqués, 19, 35–54.zbMATHGoogle Scholar
  19. Kanatani, K. (1998). Geometric information criterion for model selection. International Journal of Computer Vision (pp. 171–189).Google Scholar
  20. Kleinberg, J. M. (1999). Authorative sources in a hyberlinked environment. Journal of the ACM, 48, 604–632.MathSciNetCrossRefGoogle Scholar
  21. Minka, T. (2000). Automatic choice of dimensionality for PCA. In Neural Information Processing Systems (Vol. 13, pp. 598–604).Google Scholar
  22. Mirsky, L. (1975). A trace inequality of John von Neumann. Monatshefte für Mathematic, 79, 303–306.MathSciNetCrossRefzbMATHGoogle Scholar
  23. Pearson, K. (1901). On lines and planes of closest fit to systems of points in space. The London, Edinburgh and Dublin Philosphical Magazine and Journal of Science, 2, 559–572.CrossRefzbMATHGoogle Scholar
  24. Recht, B., Fazel, M., & Parrilo, P. (2010). Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization. SIAM Review, 52(3), 471–501.MathSciNetCrossRefzbMATHGoogle Scholar
  25. Rissanen, J. (1978). Modeling by shortest data description. Automatica, 14, 465–471.CrossRefzbMATHGoogle Scholar
  26. Shabalin, A., & Nobel, A. (2010). Reconstruction of a low-rank matrix in the presence of gaussian noise (pp. 1–34). arXiv preprint 1007.4148Google Scholar
  27. Tipping, M., & Bishop, C. (1999b). Probabilistic principal component analysis. Journal of the Royal Statistical Society, 61(3), 611–622.MathSciNetCrossRefzbMATHGoogle Scholar
  28. Turk, M., & Pentland, A. (1991). Face recognition using eigenfaces. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (pp. 586–591).Google Scholar
  29. Wallace, C., & Boulton, D. (1968). An information measure for classification. The Computer Journal, 11, 185–194.CrossRefzbMATHGoogle Scholar
  30. Wallace, C., & Dowe, D. (1999). Minimum message length and Kolmogrov complexity. The Computer Journal, 42(4), 270–283.CrossRefzbMATHGoogle Scholar

Copyright information

© Springer-Verlag New York 2016

Authors and Affiliations

  • René Vidal
    • 1
  • Yi Ma
    • 2
  • S. Shankar Sastry
    • 3
  1. 1.Center for Imaging Science Department of Biomedical EngineeringJohns Hopkins UniversityBaltimoreUSA
  2. 2.School of Information Science and Technology ShanghaiTech UniversityShanghaiChina
  3. 3.Department of Electrical Engineering and Computer ScienceUniversity of California BerkeleyBerkeleyUSA

Personalised recommendations