Effective Principal Component Analysis

  • Santosh S. Vempala
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7404)


Principal Component Analysis (PCA) is one of the most widely used algorithmic techniques. When is PCA provably effective? What are its main limitations and how can we get around them? In this note, we discuss three specific challenges.


Principal Component Analysis Singular Value Decomposition Independent Component Analysis Singular Vector Independent Component Analysis 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Achlioptas, D., McSherry, F.: On Spectral Learning of Mixtures of Distributions. In: Auer, P., Meir, R. (eds.) COLT 2005. LNCS (LNAI), vol. 3559, pp. 458–469. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  2. 2.
    Arora, S., Kannan, R.: Learning mixtures of arbitrary gaussians. Annals of Applied Probability 15(1A), 69–92 (2005)MathSciNetzbMATHCrossRefGoogle Scholar
  3. 3.
    Belkin, M., Sinha, K.: Polynomial learning of distribution families. In: FOCS, pp. 103–112 (2010)Google Scholar
  4. 4.
    Belkin, M., Sinha, K.: Toward learning gaussian mixtures with arbitrary separation. In: COLT, pp. 407–419 (2010)Google Scholar
  5. 5.
    Brubaker, S.C.: Robust pca and clustering on noisy mixtures. In: Proc. of SODA (2009)Google Scholar
  6. 6.
    Brubaker, S.C., Vempala, S.: Isotropic pca and affine-invariant clustering. In: Grötschel, M., Katona, G. (eds.) Building Bridges Between Mathematics and Computer Science. Bolyai Society Mathematical Studies, vol. 19 (2008)Google Scholar
  7. 7.
    Chaudhuri, K., Rao, S.: Learning mixtures of product distributions using correlations and independence. In: Proc. of COLT (2008)Google Scholar
  8. 8.
    Comon, P.: Independent Component Analysis. In: Proc. Int. Sig. Proc. Workshop on Higher-Order Statistics, Chamrousse, France, July 10-12, pp. 111–120 (1991); Keynote address. Republished in Lacoume, J.L. (ed.): Higher-Order Statistics, pp 29–38. Elsevier (1992)Google Scholar
  9. 9.
    DasGupta, S.: Learning mixtures of gaussians. In: Proc. of FOCS (1999)Google Scholar
  10. 10.
    DasGupta, S., Schulman, L.: A two-round variant of em for gaussian mixtures. In: Proc. of UAI (2000)Google Scholar
  11. 11.
    Eckart, C., Young, G.: The approximation of one matrix by another of lower rank. Psychometrika 1(3), 211–218 (1936)zbMATHCrossRefGoogle Scholar
  12. 12.
    Frieze, A., Jerrum, M., Kannan, R.: Learning linear transformations. In: FOCS, pp. 359–368 (1996)Google Scholar
  13. 13.
    Jutten, C., Herault, J.: Blind separation of sources, part i: An adaptive algorithm based on neuromimetic architecture. Signal Processing 24(1), 1–10 (1991)zbMATHCrossRefGoogle Scholar
  14. 14.
    Kalai, A.T., Moitra, A., Valiant, G.: Efficiently learning mixtures of two gaussians. In: STOC, pp. 553–562 (2010)Google Scholar
  15. 15.
    Kannan, R., Salmasian, H., Vempala, S.: The spectral method for general mixture models. SIAM Journal on Computing 38(3), 1141–1156 (2008)MathSciNetzbMATHCrossRefGoogle Scholar
  16. 16.
    Kannan, R., Vempala, S.: Spectral algorithms. Foundations and Trends in Theoretical Computer Science 4(3-4), 157–288 (2009)MathSciNetGoogle Scholar
  17. 17.
    Lacoume, J.-L., Ruiz, P.: Separation of independent sources from correlated inputs. IEEE Transactions on Signal Processing 40(12), 3074–3078 (1992)CrossRefGoogle Scholar
  18. 18.
    Moitra, A., Valiant, G.: Settling the polynomial learnability of mixtures of gaussians. In: FOCS, pp. 93–102 (2010)Google Scholar
  19. 19.
    Vempala, S., Wang, G.: A spectral algorithm for learning mixtures of distributions. Journal of Computer and System Sciences 68(4), 841–860 (2004)MathSciNetzbMATHCrossRefGoogle Scholar
  20. 20.
    Vempala, S., Xiao, Y.: Structure from local optima: Learning subspace juntas via higher order pca. CoRR, abs/1108.3329 (2011)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Santosh S. Vempala
    • 1
  1. 1.School of CS and Algorithms and Randomness CenterGeorgia TechAtlantaUSA

Personalised recommendations