Distributed Nonnegative Matrix Factorization with HALS Algorithm on Apache Spark

  • Krzysztof FonałEmail author
  • Rafał ZdunekEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10842)


Nonnegative Matrix Factorization (NMF) is a commonly-used unsupervised learning method for extracting parts-based features and dimensionality reduction from nonnegative data. Many computational algorithms exist for updating the latent nonnegative factors in NMF. In this study, we propose an extension of the Hierarchical Alternating Least Squares (HALS) algorithm to a distributed version using the state-of-the-art framework - Apache Spark. Spark gains its popularity among other distributed computational frameworks because of its in-memory approach which works much faster than well-known Apache Hadoop. The scalability and efficiency of the proposed algorithm is confirmed in the numerical experiments, performed on real data as well as synthetic ones.


Distributed nonnegative matrix factorization Large-scale NMF HALS algorithm Spark Recommendation systems 



This work was supported by the grant 2015/17/B/ST6/01865 funded by National Science Center (NCN) in Poland.


  1. 1.
    Jolliffe, I.T.: Principal Component Analysis. Springer Series in Statistics, 2nd edn. Springer, New York (2002). Scholar
  2. 2.
    Cichocki, A., Zdunek, R., Phan, A.H., Amari, S.I.: Nonnegative Matrix and Tensor Factorizations: Applications to Exploratory Multi-way Data Analysis and Blind Source Separation. Wiley, Hoboken (2009)CrossRefGoogle Scholar
  3. 3.
    Lee, D.D., Seung, H.S.: Learning the parts of objects by non-negative matrix factorization. Nature 401, 788–791 (1999)CrossRefGoogle Scholar
  4. 4.
    Liu, C., Yang, H.C., Fan, J., He, L.W., Wang, Y.M.: Distributed nonnegative matrix factorization for web-scale dyadic data analysis on MapReduce. In: Proceedings of 19th International Conference on World Wide Web. WWW 2010, pp. 681–690. ACM, New York (2010)Google Scholar
  5. 5.
    Sun, Z., Li, T., Rishe, N.: Large-scale matrix factorization using MapReduce. In: ICDM Workshops, pp. 1242–1248. IEEE Computer Society (2010)Google Scholar
  6. 6.
    Ding, C., Li, T., Jordan, M.I.: Convex and semi-nonnegative matrix factorizations. IEEE Trans. Pattern Anal. Mach. Intell. 32(1), 45–55 (2010)CrossRefGoogle Scholar
  7. 7.
    Cichocki, A., Zdunek, R., Amari, S.: Hierarchical ALS algorithms for nonnegative matrix and 3D tensor factorization. In: Davies, M.E., James, C.J., Abdallah, S.A., Plumbley, M.D. (eds.) ICA 2007. LNCS, vol. 4666, pp. 169–176. Springer, Heidelberg (2007). Scholar
  8. 8.
    Han, L., Neumann, M., Prasad, U.: Alternating projected Barzilai-Borwein methods for nonnegative matrix factorization. Electron. Trans. Numer. Anal. 36, 54–82 (2009–2010)Google Scholar
  9. 9.
    Kim, J., Park, H.: Fast nonnegative matrix factorization: an active-set-like method and comparisons. SIAM J. Sci. Comput. 33(6), 3261–3281 (2011)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Gillis, N., Glineur, F.: Accelerated multiplicative updates and hierarchical ALS algorithms for nonnegative matrix factorization. Neural Comput. 24(4), 1085–1105 (2012)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Chen, W., Guillaume, M.: HALS-based NMF with flexible constraints for hyperspectral unmixing. EURASIP J. Adv. Signal Process. 54, 1–14 (2012)Google Scholar
  12. 12.
    Laudadio, T., Croitor Sava, A.R., Sima, D.M., Wright, A.J., Heerschap, A., Mastronardi, N., Van Huffel, S.: Hierarchical non-negative matrix factorization applied to three-dimensional 3T MRSI data for automatic tissue characterization of the prostate. NMR Biomed. 29(6), 751–758 (2016)CrossRefGoogle Scholar
  13. 13.
    Zdunek, R., Fonal, K.: Distributed nonnegative matrix factorization with HALS algorithm on MapReduce. In: Ibrahim, S., Choo, K.-K.R., Yan, Z., Pedrycz, W. (eds.) ICA3PP 2017. LNCS, vol. 10393, pp. 211–222. Springer, Cham (2017). Scholar
  14. 14.
    Zaharia, M., Chowdhury, M., Das, T., Dave, A., Ma, J., McCauly, M., Franklin, M.J., Shenker, S., Stoica, I.: Resilient distributed datasets: a fault-tolerant abstraction for in-memory cluster computing. In: Gribble, S.D., Katabi, D. (eds.) Proceedings of the 9th USENIX Symposium on Networked Systems Design and Implementation, NSDI 2012, San Jose, CA, USA, 25–27 April 2012, pp. 15–28. USENIX Association (2012)Google Scholar
  15. 15.
    Cichocki, A., Phan, A.H.: Fast local algorithms for large scale nonnegative matrix and tensor factorizations. IEICE Trans. Fundam. Electron. Commun. Comput. Sci. E92–A(3), 708–721 (2009)CrossRefGoogle Scholar
  16. 16.
    Harper, F.M., Konstan, J.A.: The movielens datasets: history and context. ACM Trans. Interact. Intell. Syst. 5(4), 19:1–19:19 (2015)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of ElectronicsWroclaw University of TechnologyWroclawPoland

Personalised recommendations