Non-negative Matrix Factorization with Quasi-Newton Optimization
Non-negative matrix factorization (NMF) is an emerging method with wide spectrum of potential applications in data analysis, feature extraction and blind source separation. Currently, most applications use relative simple multiplicative NMF learning algorithms which were proposed by Lee and Seung, and are based on minimization of the Kullback-Leibler divergence and Frobenius norm. Unfortunately, these algorithms are relatively slow and often need a few thousands of iterations to achieve a local minimum. In order to increase a convergence rate and to improve performance of NMF, we proposed to use a more general cost function: so-called Amari alpha divergence. Taking into account a special structure of the Hessian of this cost function, we derived a relatively simple second-order quasi-Newton method for NMF. The validity and performance of the proposed algorithm has been extensively tested for blind source separation problems, both for signals and images. The performance of the developed NMF algorithm is illustrated for separation of statistically dependent signals and images from their linear mixtures.
KeywordsNonnegative Matrix Factorization Positive Matrix Factorization Nonlinear Projection Blind Source Separation Problem Negative Matrix Factorization
Unable to display preview. Download preview PDF.
- 3.Dhillon, I., Sra, S.: Generalized nonnegative matrix approximations with Bregman divergences. In: Neural Information Proc. Systems, Vancouver, Canada (2005)Google Scholar
- 4.Chu, M., Plemmons, R.J.: Nonnegative matrix factorization and applications. Bulletin of the International Linear Algebra Society 34, 2–7 (2005)Google Scholar
- 5.Pauca, V.P., Shahnaz, F., Berry, M.W., Plemmons, R.J.: Text mining using non-negative matrix factorizations. In: Proc. SIAM Inter. Conf. on Data Mining, Orlando, FL (2004)Google Scholar
- 6.Cichocki, A., Amari, S.: Adaptive Blind Signal And Image Processing (New revised and improved edition). John Wiley, New York (2003)Google Scholar
- 13.Björck, Å.: Numerical Methods for Least Squares Problems. SIAM, Philadelphia (1996)Google Scholar
- 14.Hansen, P.C.: Rank-Deficient and Discrete Ill-Posed Problems. SIAM, Philadelphia (1998)Google Scholar
- 15.Cichocki, A., Zdunek, R.: NMFLAB for Signal Processing. Technical report, Laboratory for Advanced Brain Signal Processing, BSI RIKEN, Saitama, Japan (2006), http://www.bsp.brain.riken.jp