Abstract
Non-negative matrix factorization (NMF) is a typical way for feature extraction in the framework of unsupervised learning and considers a decomposition of targeted matrix into two non-negative matrices. Conventional NMF algorithms use the Euclidean distance or the Kullback–Leibler divergence between matrix components themselves as discrepancy measures. A drawback of these conventional NMF algorithms is lack of robustness against outlier noise and these algorithms sometimes fail to extract latent structure or interpretable information from the matrix when the targeted matrix is contaminated by outlier noise. To solve the problem, we propose robust NMF algorithms by combining a statistical modeling of reconstruction and the \(\gamma \)-divergence. We theoretically investigate properties of the proposed algorithm such as convergence and robustness, and numerically show validity of the proposed algorithm.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
A short version of this article was presented as a conference paper (Machida and Takenouchi 2015). We added new theoretical analyses about robustness and convergence of empirical risk of the proposed algorithm. In addition, we added a new experiment using real data set to enhance validity of the proposed algorithm.
If a method does not have tuning parameters, both the training matrix and the validation matrix were used for training of W and H.
References
Basu, A., Harris, I. R., Hjort, N. L., & Jones, M. (1998). Robust and efficient estimation by minimising a density power divergence. Biometrika, 85(3), 549–559.
Cichocki, A., & Amari, S. (2010). Families of alpha- beta- and gamma- divergences: Flexible and robust measures of similarities. Entropy, 12(6), 1532–1568. https://doi.org/10.3390/e12061532.
Cichocki, A., Cruces, S., & Amari, S. (2011). Generalized alpha-beta divergences and their application to robust nonnegative matrix factorization. Entropy, 13(1), 134–170. Available: http://www.mdpi.com/1099-4300/13/1/134
Dikmen, O., Yang, Z., & Oja, E. (2015). Learning the information divergence. IEEE Transactions on Pattern Analysis & Machine Intelligence, 37(7), 1442–1454.
Févotte, C., & Cemgil, A. T. (2009). Nonnegative matrix factorizations as probabilistic inference in composite models. In: 17th European Signal Processing Conference. IEEE, 2009, 1913–1917.
Févotte, C., & Idier, J. (2011). Algorithms for nonnegative matrix factorization with the \(\beta \)-divergence. Neural Computation, 23(9), 2421–2456.
Févotte, C., Bertin, N., & Durrieu, J.-L. (2009). Nonnegative matrix factorization with the itakura-saito divergence: With application to music analysis. Neural Computation, 21(3), 793–830. https://doi.org/10.1162/neco.2008.04-08-771.
Fujisawa, H., & Eguchi, S. (2008). Robust parameter estimation with a small bias against heavy contamination. Journal of Multivariate Analysis, 99(9), 2053–2081. https://doi.org/10.1016/j.jmva.2008.02.004.
Kanamori, T., & Fujisawa, H. (2015). Robust estimation under heavy contamination using unnormalized models. Biometrika, 102(3), 559–572.
Lee, D. D., & Seung, H. S. (1999). Learning the parts of objects by nonnegative matrix factorization. Nature, 401, 788–791.
Lee, D. D., & Seung, H. S. (2001). “Algorithms for non-negative matrix factorization,” In: Advances in Neural Information Processing Systems, pp. 556–562.
Machida, K., & Takenouchi, T. (2015). Non-negative matrix factorization based on \(\gamma \)-divergence. In :International Joint Conference on Neural Networks (IJCNN) . IEEE, pp. 1–6.
Minka, T. (2005). Divergence measures and message passing. Technical report, Microsoft Research.
Yang, Z., & Oja, E. (2011). Unified development of multiplicative algorithms for linear and quadratic nonnegative matrix factorization. IEEE Transactions on Neural Networks, 22(12), 1878–1891.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
On behalf of all authors, the corresponding author states that there is no conflict of interest.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
This work was supported by JSPS KAKENHI Grant Numbers 16K00051 from MEXT, Japan.
Rights and permissions
About this article
Cite this article
Machida, K., Takenouchi, T. Statistical modeling of robust non-negative matrix factorization based on \(\gamma \)-divergence and its applications. Jpn J Stat Data Sci 2, 441–464 (2019). https://doi.org/10.1007/s42081-019-00041-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s42081-019-00041-3