Abstract
We tackle the parallelization of Non-Negative Matrix Factorization (NNMF), using the Alternating Least Squares and Lee and Seung algorithms, motivated by its use in audio source separation. For the first algorithm, a very suitable technique is the use of active set algorithms for solving several non-negative inequality constraints least squares problems. We have addressed the NNMF for dense matrix on multicore architectures, by organizing these optimization problems for independent columns. Although in the sequential case, the method is not as efficient as the block pivoting variant used by other authors, they are very effective in the parallel case, producing satisfactory results for the type of applications where is to be used. For the Lee and Seung method, we propose a reorganization of the algorithm steps that increases the convergence speed and a parallelization of the solution. The article also includes a theoretical and experimental study of the performance obtained with similar matrices to that which arise in applications that have motivated this work.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Lee DD, Seung HS (1999) Learning the parts of objects by non-negative matrix factorization. Nature 401:788–791
Lee DD, Seung HS (2001) Algorithms for non-negative matrix factorization. Advances in neural information processing systems, vol 13. MIT Press, Cambridge, pp 556–562
Paatero P, Tapper U (1994) Positive matrix factorization: a non-negative factor model with optimal utilization of error estimates of data values. Environmetrics 5:111–126
Paatero P (1997) Least squares formulation of robust non-negative factor analysis. Chemometr Intell. Lab. Syst. 37:23–35
Battenberg E, Freed A, Wessel D (2010) Advances in the parallelization of music and audio applications. International computer music conference. Stony Brook, New York
Wnag J, Zhong W, Zhang J (2006) NNMF-based factorization techniques for high-accuracy privacy protection on non-negative-valued datasets, data mining workshops, 2006. ICDM Workshops 2006. Sixth IEEE International Conference on Computing and Processing, pp 513–517
Rodriguez-Serrano FJ, Carabias-Orti JJ, Vera-Candeas P, Virtanen T, Ruiz-Reyes N (2012) Multiple instrument mixtures source separation evaluation using instrument-dependent NMF models. In: Theis F (ed) LVA/ICA 2012. Springer-Verlag, Berlin, pp 380–387
Xu W, Liu X, Gong Y (2003) Document clustering based on non-negative matrix factorization, Proceedings of the 26th annual international ACM SIGIR conference on Research and development in information retrieval. SIGIR03, Toronto, pp 267–273.
Berry MW, Browne M, Langville A, Pauca V, Plemmons R (2007) Algorithms and applications for aproximate nonnegative matrix factorization. Comput Stat Data Anal 52:155–173
Kim J, Park H (2007) Sparse non-negative matrix factorizations via alternating non-negativity-constrained least squares for microarray data analysis. Bioinformatics 23:1495–1502
Devajaran K (2008) Nonnegative matrix factorization: an analytical and interpretative tool in computational biology. PLoS Comput Biol 4(7):e1000029
Guan N, Tao D, Luo Z, Yuan B (2012) NeNMF: an optimal gradient method for nonnegative matrix factorization. IEEE Trans Signal Proc 60(6):2882–2898
Cichocki A, Phan A-H (2009) Fast local algorithms for large scale nonnegative matrix and tensor factorizations. IEICE Trans Fund Electron Commun Comput Sci E92– A:708–721
Cichocki A, Zdunek R, Amari S-I (2007) Hierarchical ALS algorithms for nonnegative matrix and 3d tensor factorization, independent component analysis and signal separation, lecture notes in computer science, vol 4666. Springer, NewYork, pp 169–176
Cichocki A, Cruces S, Amari S-I (2011) Generalized alpha–beta divergences and their application to robust nonnegative matrix factorization. Entropy 13:134–170
Kim J, Park H (2011) Fast nonnegative matrix factorization: an active-set-like method and comparisons. SIAM J Sci Comput 33(6):3261–3281
Kim D, Sra S, Dhillon IS (2007) Fast newton-type methods for the least squares nonnegative matrix approximation problem, Proceedings of the seventh SIAM international conference on data mining. SIAM, Philadelphia, pp 343–354
Kim J, Park H (2008) Nonnegative matrix factorization based on alternating nonnegativity constrained least squares and active set method. SIAM J Matrix Anal Appl 30:713–730
Carabias-Orti JJ, Virtanen T, Vera-Candeas P, Ruiz-Reyes N, Caadas-Quesada FJ (2011) Musical instrument sound multi-excitation model for non-negative spectrogram factorization. IEEE J Select Top Signal Proc 5(6):1144–1158
Mejia-Roa E, Garcia C, Gomez JI, Prieto M, Tenllado C, Pascual-Montano A, Tirado F (2012) Parallelism on the non-negative matrix factorization, applications, tools and techniques on the road to exascale computing, vol 22. IOS Press, Netherlands
Dong C, Zhoa H, Wang W (2010) Parallel nonnegative matrix factorization algorithm on the distributed memory platform. Int J Par Prog 38:117–137
Grippo L, Sciandrone M (2000) On the convergence of the block nonlinear Gauss–Seidel method under convex constraints. Oper Res Lett 26:127–136
Lawson CL, Hanson RJ (1995) Solving least squares problems. SIAM, Philadelphia
Bjorck A (1996) Numerical methods for least squares problems. SIAM, Philadelphia
MATLAB version 8.0 (R2012b) (2012) The MathWorks Inc, Natick, Massachusetts.
Acknowledgments
This work has been supported by European Union ERDF and Spanish Government through TEC2012-38142-C04 project and Generalitat Valenciana through PROMETEO/2009/013 project.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Alonso, P., García, V.M., Martínez-Zaldívar, F.J. et al. Parallel approach to NNMF on multicore architecture. J Supercomput 70, 564–576 (2014). https://doi.org/10.1007/s11227-013-1083-8
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11227-013-1083-8