A Parallel Non-negative Sparse Large Matrix Factorization

  • Anatoly Anisimov
  • Oleksandr Marchenko
  • Emil Nasirov
  • Stepan Palamarchuk
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8686)


This paper proposes parallel methods of non-negative large sparse matrix factorization – a very popular technique in computational linguistics. Memory usage and data transmitting necessity of factorization algorithm was analysed and optimized. The described effective GPU-based and distributed algorithms were implemented, tested and compared by means of large sparse matrices processing.


Computational linguistics parallel computations non- negative matrix factorization 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Deerwester, S., Dumais, S.T., Furnas, G.W., Landauer, T.K., Harshman, R.: Indexing by latent semantic analysis. Journal of the American Society for Information Science, 391–407 (1990)Google Scholar
  2. 2.
    Xu, W., Liu, X., Gong, Y.: Document Clustering Based on Non-negative Matrix Factorization. In: Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Informaion Retrieval, SIGIR 2003, pp. 267–273. ACM, New York (2003)CrossRefGoogle Scholar
  3. 3.
    Shahnaz, F., Berry, M.W., Pauca, V.P., Plemmons, R.J.: Document Clustering Using Nonnegative Matrix Factorization. Inf. Process. Manage., 373–386 (2006)Google Scholar
  4. 4.
    Landauer, T.K., Foltz, P.W., Laham, D.: An introduction to latent semantic analysis Discourse processes, pp. 259–284. Ablex Publishing Co. (1998)Google Scholar
  5. 5.
    Mihalcea, R., Corley, C., Strapparava, C.: Corpus-based and knowledge-based measures of text semantic similarity. In: AAAI 2006, pp. 775–780. AAAI Press, Menlo Park (2006)Google Scholar
  6. 6.
    Van de Cruys, T.: A non-negative tensor factorization model for selectional preference induction. Natural Language Engineering, 417–437 (2010)Google Scholar
  7. 7.
    Bader, B.W., Kolda, T.G.: MATLAB Tensor Toolbox Version 2.5 (2012),
  8. 8.
    Kanjani, K.: Parallel Non Negative Matrix Factorization for Document Clustering Texas A & M University (2007)Google Scholar
  9. 9.
    Kysenko, V., Rupp, K., Marchenko, O., Selberherr, S., Anisimov, A.: GPU-Accelerated Non-negative Matrix Factorization for Text Mining. In: Bouma, G., Ittoo, A., Métais, E., Wortmann, H. (eds.) NLDB 2012. LNCS, vol. 7337, pp. 158–163. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  10. 10.
    Liu, C., Yang, H.-C., Fan, J., He, L.-W., Wang, Y.-M.: Distributed Nonnegative Matrix Factorization for Web-scale Dyadic Data Analysis on Mapreduce. In: Proceedings of the 19th International Conference on World Wide Web, WWW 2010, pp. 681–690. ACM, Raleigh (2010)CrossRefGoogle Scholar
  11. 11.
    Lee, D.D., Seung, H.S.: Algorithms for Non-negative Matrix Factorization In NIPS, pp. 556–562. MIT Press (2000)Google Scholar
  12. 12.
    Naumov, M., Chien, L.S., Vandermersch, P., Kapasi, U.: CUDA CUSPARSE Library NVIDIA, San Jose, CA (2010)Google Scholar
  13. 13.
    NVIDIA: CUBLAS Library User Guide (2013),
  14. 14.
    Cickocki, A., Zdunek, R., Phan, A.H., Amari, S.-I.: Non-negative matrix and tensor factorizations: applications to exploratory multiway data analysis and blind source separation Fabulous, Singapore, pp. 237–240 (2009)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Anatoly Anisimov
    • 1
  • Oleksandr Marchenko
    • 1
  • Emil Nasirov
    • 1
  • Stepan Palamarchuk
    • 1
  1. 1.Faculty of CyberneticsTaras Shevchenko National University of KyivUkraine

Personalised recommendations