Initialization of Nonnegative Matrix Factorization with Vertices of Convex Polytope

  • Rafal Zdunek
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7267)

Abstract

Nonnegative Matrix Factorization (NMF) is an emerging unsupervised learning technique that has already found many applications in machine learning and multivariate nonnegative data processing. NMF problems are usually solved with an alternating minimization of a given cost function, which leads to non-convex optimization. For this approach, an initialization for the factors to be estimated plays an essential role, not only for a fast convergence rate but also for selection of the desired local minima. If the observations are modeled by the exact factorization model (consistent data), NMF can be easily obtained by finding vertices of the convex polytope determined by the observed data projected on the probability simplex. For an inconsistent case, this model can be relaxed by approximating mean localizations of the vertices. In this paper, we discuss these issues and propose the initialization algorithm based on the analysis of a geometrical structure of the observed data. This approach is demonstrated to be robust, even for moderately noisy data.

Keywords

Monte Carlo Blind Source Separation Nonnegative Matrix Factorization Convex Polytope Probability Simplex 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Lee, D.D., Seung, H.S.: Learning the parts of objects by non-negative matrix factorization. Nature 401, 788–791 (1999)CrossRefGoogle Scholar
  2. 2.
    Langville, A.N., Meyer, C.D., Albright, R.: Initializations for the nonnegative matrix factorization. In: Proc. of the Twelfth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Philadelphia, USA (2006)Google Scholar
  3. 3.
    Cichocki, A., Zdunek, R., Phan, A.H., Amari, S.I.: Nonnegative Matrix and Tensor Factorizations: Applications to Exploratory Multi-way Data Analysis and Blind Source Separation. Wiley and Sons (2009)Google Scholar
  4. 4.
    Cichocki, A., Zdunek, R.: Multilayer nonnegative matrix factorization. Electronics Letters 42(16), 947–948 (2006)CrossRefGoogle Scholar
  5. 5.
    Wild, S.: Seeding non-negative matrix factorization with the spherical k-means clustering. M.Sc. Thesis, University of Colorado (2000)Google Scholar
  6. 6.
    Wild, S., Curry, J., Dougherty, A.: Improving non-negative matrix factorizations through structured initialization. Pattern Recognition 37(11), 2217–2232 (2004)CrossRefGoogle Scholar
  7. 7.
    Boutsidis, C., Gallopoulos, E.: SVD based initialization: A head start for nonnegative matrix factorization. Pattern Recognition 41, 1350–1362 (2008)MATHCrossRefGoogle Scholar
  8. 8.
    Kim, Y.D., Choi, S.: A method of initialization for nonnegative matrix factorization. In: Proc. IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2007), Honolulu, Hawaii, USA, vol. II, pp. 537–540 (2007)Google Scholar
  9. 9.
    Xue, Y., Tong, C.S., Chen, Y., Chen, W.S.: Clustering-based initialization for non-negative matrix factorization. Applied Mathematics and Computation 205(2), 525–536 (2008); Special Issue on Advanced Intelligent Computing Theory and Methodology in Applied Mathematics and ComputationMathSciNetMATHCrossRefGoogle Scholar
  10. 10.
    Donoho, D., Stodden, V.: When does non-negative matrix factorization give a correct decomposition into parts? In: Thrun, S., Saul, L., Schölkopf, B. (eds.) Advances in Neural Information Processing Systems (NIPS), vol. 16. MIT Press, Cambridge (2004)Google Scholar
  11. 11.
    Chu, M.T., Lin, M.M.: Low dimensional polytype approximation and its applications to nonnegative matrix factorization. SIAM Journal of Scientific Computing 30, 1131–1151 (2008)MathSciNetMATHCrossRefGoogle Scholar
  12. 12.
    Wang, F.Y., Chi, C.Y., Chan, T.H., Wang, Y.: Nonnegative least-correlated component analysis for separation of dependent sources by volume maximization. IEEE Transactions Pattern Analysis and Machine Intelligence 32(5), 875–888 (2010)CrossRefGoogle Scholar
  13. 13.
    Cichocki, A., Zdunek, R.: NMFLAB for Signal and Image Processing. Technical report, Laboratory for Advanced Brain Signal Processing, BSI, RIKEN, Saitama, Japan (2006)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Rafal Zdunek
    • 1
  1. 1.Institute of Telecommunications, Teleinformatics and AcousticsWroclaw University of TechnologyWroclawPoland

Personalised recommendations