Multifrontal Non-negative Matrix Factorization

  • Piyush SaoEmail author
  • Ramakrishnan Kannan
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12043)


Non-negative matrix factorization (Nmf) is an important tool in high-performance large scale data analytics with applications ranging from community detection, recommender system, feature detection and linear and non-linear unmixing. While traditional Nmf works well when the data set is relatively dense, however, it may not extract sufficient structure when the data is extremely sparse. Specifically, traditional Nmf fails to exploit the structured sparsity of the large and sparse data sets resulting in dense factors. We propose a new algorithm for performing Nmf on sparse data that we call multifrontal Nmf (Mf-Nmf) since it borrows several ideas from the multifrontal method for unconstrained factorization (e.g. LU and QR). We also present an efficient shared memory parallel implementation of Mf-Nmf and discuss its performance and scalability. We conduct several experiments on synthetic and realworld datasets and demonstrate the usefulness of the algorithm by comparing it against standard baselines. We obtain a speedup of 1.2x to 19.5x on 24 cores with an average speed up of 10.3x across all the real world datasets.


Sparse matrix computations Non-negative matrix factorization Data analysis Multifrontal methods 


  1. 1.
    Cichocki, A., Zdunek, R., Phan, A.H., Amari, S.: Nonnegative Matrix and Tensor Factorizations: Applications to Exploratory Multi-way Data Analysis and Blind Source Separation. Wiley, Chichester (2009)CrossRefGoogle Scholar
  2. 2.
    Davis, T.: Multifrontral multithreaded rank-revealing sparse QR factorization. In: Dagstuhl Seminar Proceedings. Schloss Dagstuhl-Leibniz-Zentrum für Informatik (2009)Google Scholar
  3. 3.
    Duff, I.S., Erisman, A.M., Reid, J.K.: Direct Methods for Sparse Matrices. Clarendon Press, Oxford (1986)zbMATHGoogle Scholar
  4. 4.
    Fairbanks, J.P., Kannan, R., Park, H., Bader, D.A.: Behavioral clusters in dynamic graphs. Parallel Comput. 47, 38–50 (2015)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Flatz, M., Kutil, R., Vajteršic, M.: Parallelization of the hierarchical alternating least squares algorithm for nonnegative matrix factorization. In: 2018 IEEE 4th International Forum on Research and Technology for Society and Industry (RTSI), pp. 1–5. IEEE (2018)Google Scholar
  6. 6.
    Flatz, M., Vajteršic, M.: A parallel algorithm for nonnegative matrix factorization based on Newton iteration. In: Proceedings of the IASTED International Conference Parallel and Distributed Computing and Networks (PDCN 2013), pp. 600–607. ACTA Press (2013)Google Scholar
  7. 7.
    Gemulla, R., Nijkamp, E., Haas, P.J., Sismanis, Y.: Large-scale matrix factorization with distributed stochastic gradient descent. In: Proceedings of the KDD, pp. 69–77. ACM (2011)Google Scholar
  8. 8.
    George, A.: Nested dissection of a regular finite element mesh. SIAM J. Numer. Anal. 10(2), 345–363 (1973)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Grigori, L., Boman, E.G., Donfack, S., Davis, T.A.: Hypergraph-based unsymmetric nested dissection ordering for sparse LU factorization. SIAM J. Sci. Comput. 32(6), 3426–3446 (2010)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Guan, N., Tao, D., Luo, Z., Yuan, B.: NeNMF: an optimal gradient method for nonnegative matrix factorization. IEEE Trans. Signal Process. 60(6), 2882–2898 (2012)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Heath, M.T., Ng, E., Peyton, B.W.: Parallel algorithms for sparse linear systems. SIAM Rev. 33(3), 420–460 (1991)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Ho, N.-D., Van Dooren, P., Blondel, V.D.: Descent methods for nonnegative matrix factorization. CoRR, abs/0801.3199 (2008)Google Scholar
  13. 13.
    Kannan, R., Ballard, G., Park, H.: MPI-FAUN: an MPI-based framework for alternating-updating nonnegative matrix factorization. IEEE Trans. Knowl. Data Eng. 30(3), 544–558 (2018)CrossRefGoogle Scholar
  14. 14.
    Kim, J., He, Y., Park, H.: Algorithms for nonnegative matrix and tensor factorizations: a unified view based on block coordinate descent framework. J. Global Optim. 58(2), 285–319 (2014)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Kim, J., Park, H.: Fast nonnegative matrix factorization: an active-set-like method and comparisons. SIAM J. Sci. Comput. 33(6), 3261–3281 (2011)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Kim, W., Chen, B., Kim, J., Pan, Y., Park, H.: Sparse nonnegative matrix factorization for protein sequence motif discovery. Expert Syst. Appl. 38(10), 13198–13207 (2011)CrossRefGoogle Scholar
  17. 17.
    Lee, D.D., Sebastian Seung, H.: Algorithms for non-negative matrix factorization. In: NIPS, vol. 13, pp. 556–562 (2001)Google Scholar
  18. 18.
    Liao, R., Zhang, Y., Guan, J., Zhou, S.: CloudNMF: a MapReduce implementation of nonnegative matrix factorization for large-scale biological datasets. Genomics Proteomics Bioinform. 12(1), 48–51 (2014)CrossRefGoogle Scholar
  19. 19.
    Liu, C., Yang, H.C., Fan, J., He, L.-W., Wang, Y.-M.: Distributed nonnegative matrix factorization for web-scale dyadic data analysis on MapReduce. In: Proceedings of the WWW, pp. 681–690. ACM (2010)Google Scholar
  20. 20.
    Sao, P., Li, X.S., Vuduc, R.: A communication-avoiding 3D LU factorization algorithm for sparse matrices. In: Proceedings of the IEEE International Parallel and Distributed Processing Symposium (IPDPS), Vancouver, BC, Canada, May 2018Google Scholar
  21. 21.
    Sao, P., Vuduc, R., Li, X.S.: A distributed CPU-GPU sparse direct solver. In: Silva, F., Dutra, I., Santos Costa, V. (eds.) Euro-Par 2014. LNCS, vol. 8632, pp. 487–498. Springer, Cham (2014). Scholar
  22. 22.
    Sun, D.L., Févotte, C.: Alternating direction method of multipliers for non-negative matrix factorization with the beta-divergence. In: 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 6201–6205, May 2014Google Scholar
  23. 23.
    Yin, J., Gao, L., Zhang, Z.M.: Scalable nonnegative matrix factorization with block-wise updates. In: Calders, T., Esposito, F., Hüllermeier, E., Meo, R. (eds.) ECML PKDD 2014. LNCS (LNAI), vol. 8726, pp. 337–352. Springer, Heidelberg (2014). Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Oak Ridge National LaboratoryOak RidgeUSA

Personalised recommendations