Advances in Incremental PCA Algorithms

  • Tal Halpern
  • Sivan ToledoEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10777)


We present a range of new incremental (single-pass streaming) algorithms for incremental principal components analysis (IPCA) and show that they are more effective than exiting ones. IPCA algorithms process the columns of a matrix A one at a time and attempt to build a basis for a low-dimensional subspace that spans the dominant subspace of A. We present a unified framework for IPCA algorithms, show that many existing ones are parameterizations of it, propose new sophisticated algorithms, and show that both the new algorithms and many existing ones can be implemented more efficiently than was previously known. We also show that many existing algorithms can fail even in easy cases and we show experimentally that our new algorithms outperform existing ones.


Principal components analysis Streaming algorithms Frequent directions 


  1. 1.
    Brand, M.: Incremental singular value decomposition of uncertain data with missing values. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2350, pp. 707–720. Springer, Heidelberg (2002). CrossRefGoogle Scholar
  2. 2.
    Brand, M.: Fast low-rank modifications of the thin singular value decomposition. Linear Algebra Appl. 415(1), 20–30 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Chahlaoui, Y., Gallivan, K.A., Dooren, P.V.: An incremental method for computing dominant singular spaces. In: Computational Information Retrieval, pp. 53–62 (2001)Google Scholar
  4. 4.
    Chahlaoui, Y., Gallivan, K.A., Van Dooren, P.: Recursive calculation of dominant singular subspaces. SIAM J. Matrix Anal. Appl. 25(2), 445–463 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Chandrasekaran, S., Manjunath, B., Wang, Y.-F., Winkeler, J., Zhang, H.: An eigenspace update algorithm for image analysis. Graph. Models Image Process. 59(5), 321–332 (1997)CrossRefGoogle Scholar
  6. 6.
    Desai, A., Ghashami, M., Phillips, J.M.: Improved practical matrix sketching with guarantees. IEEE Trans. Knowl. Data Eng. 28(7), 1678–1690 (2016)CrossRefzbMATHGoogle Scholar
  7. 7.
    Ghashami, M., Liberty, E., Phillips, J.M., Woodruff, D.P.: Frequent directions: simple and deterministic matrix sketching. SIAM J. Comput. 45(5), 1762–1792 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Halpern, T.: Fast and robust algorithms for large-scale streaming PCA. Master’s thesis, Tel Aviv University, July 2017.
  9. 9.
    Levey, A., Lindenbaum, M.: Sequential Karhunen-Loeve basis extraction and its application to images. IEEE Trans. Image Process. 9(8), 1371–1374 (2000)CrossRefzbMATHGoogle Scholar
  10. 10.
    Liberty, E.: Simple and deterministic matrix sketching. In: Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 581–588. ACM (2013)Google Scholar
  11. 11.
    Manjunath, B., Chandrasekaran, S., Wang, Y.-F.: An eigenspace update algorithm for image analysis. In: Proceedings International Symposium on Computer Vision, pp. 551–556. IEEE (1995)Google Scholar
  12. 12.
    O’Brien, G.W.: Information management tools for updating an SVD-encoded indexing scheme. Master’s thesis, University of Tennessee, Knoxville (1994)Google Scholar
  13. 13.
    Roweis, S.T., Saul, L.K.: Nonlinear dimensionality reduction by locally linear embedding. Sci. 290(5500), 2323–2326 (2000)CrossRefGoogle Scholar
  14. 14.
    Tenenbaum, J.B., De Silva, V., Langford, J.C.: A global geometric framework for nonlinear dimensionality reduction. Sci. 290(5500), 2319–2323 (2000)CrossRefGoogle Scholar
  15. 15.
    Wah, C., Branson, S., Welinder, P., Perona, P., Belongie, S.: The Caltech-UCSD Birds-200-2011 Dataset. Technical report CNS-TR-2011-001, California Institute of Technology (2011).
  16. 16.
    Zha, H., Simon, H.D.: On updating problems in latent semantic indexing. SIAM J. Sci. Comput. 21(2), 782–791 (1999)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Tel-Aviv UniversityTel AvivIsrael

Personalised recommendations