Abstract
Non-negative Matrix Factorization (NMF), especially with sparseness constraints, plays a critically important role in data engineering and machine learning. Hoyer (2004) presented an algorithm to compute NMF with exact sparseness constraints. The exact sparseness constraints depends on a projection operator. In the present work, we first give a very simple counterexample, for which the projection operator of the Hoyer (2004) algorithm fails. After analysing the reason geometrically, we fix this bug by adding some random terms and show that the fixed one works correctly. Based on the fixed projection operator, we propose another sparse NMF algorithm aiming at optimizing the generalized Kullback-Leibler divergence, hence named SNMF-GKLD. Experimental results show that SNMF-GKLD not only has similar effects with Hoyer (2004) on the same data sets, but is also efficient.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Berry, M.W., Browne, M., Langville, A.N., Pauca, V.P., Plemmons, R.J.: Algorithms and applications for approximate nonnegative matrix factorization. Comput. Stat. Data Anal. 52(1), 155–173 (2007)
Boutsidis, C., Gallopoulos, E.: SVD based initialization: a head start for nonnegative matrix factorization. Pattern Recogn. 41(4), 1350–1362 (2008)
Cichocki, A., Amari, S.I., Zdunek, R., Kompass, R., Hori, G., He, Z.: Extended SMART algorithms for non-negative matrix factorization. In: Rutkowski, L., Tadeusiewicz, R., Zadeh, L.A., Żurada, J.M. (eds.) ICAISC 2006. Lecture Notes in Artificial Intelligence (LNAI), vol. 4029, pp. 548–562. Springer, Heidelberg (2006). doi:10.1007/11785231_58
Cichocki, A., Zdunek, R., Phan, A.H., Amari, S.I.: Nonnegative Matrix and Tensor Factorizations: Applications to Exploratory Multi-way Data Analysis and Blind Source Separation. John Wiley & Sons Ltd., Chichester (2009)
Hoyer, P.O.: Non-negative sparse coding. In: Bourlard, H., Adali, T., Bengio, S., Larsen, J., Douglas, S. (eds.) NNSP 2012, pp. 557–565. IEEE, New York (2002)
Hoyer, P.O.: Non-negative matrix factorization with sparseness constraints. J. Mach. Learn. Res. 5, 1457–1469 (2004)
Huang, Z., Zhou, A., Zhang, G.: Non-negative matrix factorization: a short survey on methods and applications. In: Li, K., Li, J., Liu, Y., Castiglione, A. (eds.) ISICA 2015. CCIS, vol. 575, pp. 331–340. Springer, Heidelberg (2012). doi:10.1007/978-3-642-34289-9_37
Hyvärinen, A., Karhunen, J., Oja, E.: Independent Component Analysis. John Wiley & Sons, New York (2001)
Jolliffe, I.T.: Principal Component Analysis, 2nd edn. Springer, New York (2002)
Kim, H., Park, H.: Sparse non-negative matrix factorizations via alternating non-negativity-constrained least squares for microarray data analysis. Bioinformatics 23(12), 1495–1502 (2007)
Lee, D.D., Seung, H.S.: Learning the parts of objects by non-negative matrix factorization. Nature 401(6755), 788–791 (1999)
Lee, D.D., Seung, H.S.: Algorithms for non-negative matrix factorization. In: Leen, T.K., Dietterich, T.G., Tresp, V. (eds.) NIPS*2000, pp. 556–562. MIT Press, Cambridge (2001)
Li, S.Z., Hou, X., Zhang, H., Cheng, Q.: Learning spatially localized, parts-based representation. In: CVPR 2001, vol. 1, pp. 207–212. IEEE, Los Alamitos (2001)
Liu, W., Zheng, N.: Learning sparse features for classification by mixture models. Pattern Recogn. Lett. 25(2), 155–161 (2004)
Liu, W., Zheng, N., Lu, X.: Non-negative matrix factorization for visual coding. In: ICASSP 2003, vol. 3, pp. 293–296. IEEE, Piscataway (2003)
Pascual-Montano, A., Carazo, J.M., Kochi, K., Lehmann, D., Pascual-Marqui, R.D.: Nonsmooth nonnegative matrix factorization (nsNMF). IEEE Trans. Pattern Anal. Mach. Intell. 28(3), 403–415 (2006)
Stadlthanner, K., Theis, F.J., Puntonet, C.G., Lang, E.W.: Extended sparse nonnegative matrix factorization. In: Cabestany, J., Prieto, A., Sandoval, F. (eds.) IWANN 2005. LNCS, vol. 3512, pp. 249–256. Springer, Heidelberg (2005). doi:10.1007/11494669_31
Tong, M., Guo, J., Tao, S., Wu, Y.: Independent detection and self-recovery video authentication mechanism using extended NMF with different sparseness constraints. Multimedia Tools Appl. 75(13), 8045–8069 (2016)
Acknowledgments
This work was partially supported by NSFC (11471307, 11501540, 61572024), CAS “Light of West China” Program (2014), NSF of Hunan Province (2015JJ3071) and Chongqing Research Program (cstc2015jcyjys40001).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing AG
About this paper
Cite this paper
Chen, J., Feng, Y., Liu, Y., Tang, B., Wu, W. (2016). Sparse Non-negative Matrix Factorization with Generalized Kullback-Leibler Divergence. In: Yin, H., et al. Intelligent Data Engineering and Automated Learning – IDEAL 2016. IDEAL 2016. Lecture Notes in Computer Science(), vol 9937. Springer, Cham. https://doi.org/10.1007/978-3-319-46257-8_38
Download citation
DOI: https://doi.org/10.1007/978-3-319-46257-8_38
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-46256-1
Online ISBN: 978-3-319-46257-8
eBook Packages: Computer ScienceComputer Science (R0)