Abstract
We propose a new variant of nonnegative matrix factorization (NMF), combining separability and sparsity assumptions. Separability requires that the columns of the first NMF factor are equal to columns of the input matrix, while sparsity requires that the columns of the second NMF factor are sparse. We call this variant sparse separable NMF (SSNMF), which we prove to be NP-complete, as opposed to separable NMF which can be solved in polynomial time. The main motivation to consider this new model is to handle underdetermined blind source separation problems, such as multispectral image unmixing. We introduce an algorithm to solve SSNMF, based on the successive nonnegative projection algorithm (SNPA, an effective algorithm for separable NMF), and an exact sparse nonnegative least squares solver. We prove that, in noiseless settings and under mild assumptions, our algorithm recovers the true underlying sources. This is illustrated by experiments on synthetic data sets and the unmixing of a multispectral image.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
It stands for brassens Relies on Assumptions of Separability and Sparsity for Elegant NMF Solving.
- 2.
References
Araújo, M.C.U., Saldanha, T.C.B., Galvão, R.K.H., Yoneyama, T., Chame, H.C., Visani, V.: The successive projections algorithm for variable selection in spectroscopic multicomponent analysis. Chemometr. Intell. Lab. Syst. 57(2), 65–73 (2001)
Arora, S., Ge, R., Kannan, R., Moitra, A.: Computing a nonnegative matrix factorization – provably. In: Proceedings of the Forty-Fourth Annual ACM Symposium on Theory of Computing, pp. 145–162 (2012)
Bioucas-Dias, J.M., Plaza, A., Dobigeon, N., Parente, M., Du, Q., Gader, P., Chanussot, J.: Hyperspectral unmixing overview: geometrical, statistical, and sparse regression-based approaches. IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens. 5(2), 354–379 (2012)
Cohen, J.E., Gillis, N.: Nonnegative low-rank sparse component analysis. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 8226–8230 (2019)
El Ghaoui, L., Viallon, V., Rabbani, T.: Safe feature elimination in sparse supervised learning technical report no. Technical report, UC/EECS-2010-126, EECS Department, University of California at Berkeley (2010)
Fu, X., Huang, K., Sidiropoulos, N.D., Ma, W.K.: Nonnegative matrix factorization for signal and data analytics: identifiability, algorithms, and applications. IEEE Signal Process. Mag. 36(2), 59–80 (2019)
Gillis, N.: Successive nonnegative projection algorithm for robust nonnegative blind source separation. SIAM J. Imag. Sci. 7, 1420–1450 (2014)
Gillis, N.: The why and how of nonnegative matrix factorization. Regularization, optimization, kernels, and support vector machines 12(257), 257–291 (2014)
Hoyer, P.O.: Non-negative sparse coding. In: Proceedings of the 12th IEEE Workshop on Neural Networks for Signal Processing, pp. 557–565 (2002)
Hoyer, P.O.: Non-negative matrix factorization with sparseness constraints. J. Mach. Learn. Res. 5, 1457–1469 (2004)
Kim, H., Park, H.: Sparse non-negative matrix factorizations via alternating non-negativity-constrained least squares for microarray data analysis. Bioinformatics 23(12), 1495–1502 (2007)
Kumar, A., Sindhwani, V., Kambadur, P.: Fast conical hull algorithms for near-separable non-negative matrix factorization. In: Proceedings of the 30th International Conference on Machine Learning (2013)
Lee, D.D., Seung, H.S.: Learning the parts of objects by non-negative matrix factorization. Nature 401(6755), 788–791 (1999)
Ma, W.K., et al.: A signal processing perspective on hyperspectral unmixing: insights from remote sensing. IEEE Signal Process. Mag. 31(1), 67–81 (2014)
Naanaa, W., Nuzillard, J.M.: Blind source separation of positive and partially correlated data. Sig. Process. 85(9), 1711–1722 (2005)
Nadisic, N., Vandaele, A., Gillis, N., Cohen, J.E.: Exact sparse nonnegative least squares. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 5395–5399 (2020)
Natarajan, B.K.: Sparse approximate solutions to linear systems. SIAM J. Comput. 24(2), 227–234 (1995)
Sun, Y., Xin, J.: Underdetermined sparse blind source separation of nonnegative and partially overlapped data. SIAM J. Sci. Comput. 33(4), 2063–2094 (2011)
Vavasis, S.A.: On the complexity of nonnegative matrix factorization. SIAM J. Optim. 20(3), 1364–1377 (2010)
Zhu, F.: Hyperspectral unmixing: ground truth labeling, datasets, benchmark performances and survey. arXiv preprint arXiv:1708.05125 (2017)
Zhu, F., Wang, Y., Xiang, S., Fan, B., Pan, C.: Structured sparse method for hyperspectral unmixing. ISPRS J. Photogram. Remote Sens. 88, 101–118 (2014)
Acknowledgments
The authors are grateful to the reviewers, whose insightful comments helped improve the paper. NN and NG acknowledge the support by the European Research Council (ERC starting grant No 679515), and by the Fonds de la Recherche Scientifique - FNRS and the Fonds Wetenschappelijk Onderzoek - Vlanderen (FWO) under EOS project O005318F-RG47.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Nadisic, N., Vandaele, A., Cohen, J.E., Gillis, N. (2021). Sparse Separable Nonnegative Matrix Factorization. In: Hutter, F., Kersting, K., Lijffijt, J., Valera, I. (eds) Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2020. Lecture Notes in Computer Science(), vol 12457. Springer, Cham. https://doi.org/10.1007/978-3-030-67658-2_20
Download citation
DOI: https://doi.org/10.1007/978-3-030-67658-2_20
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-67657-5
Online ISBN: 978-3-030-67658-2
eBook Packages: Computer ScienceComputer Science (R0)
