Projective Nonnegative Matrix Factorization for Image Compression and Feature Extraction
In image compression and feature extraction, linear expansions are standardly used. It was recently pointed out by Lee and Seung that the positivity or non-negativity of a linear expansion is a very powerful constraint, that seems to lead to sparse representations for the images. Their technique, called Non-negative Matrix Factorization (NMF), was shown to be a useful technique in approximating high dimensional data where the data are comprised of non-negative components. We propose here a new variant of the NMF method for learning spatially localized, sparse, part-based subspace representations of visual patterns. The algorithm is based on positively constrained projections and is related both to NMF and to the conventional SVD or PCA decomposition. Two iterative positive projection algorithms are suggested, one based on minimizing Euclidean distance and the other on minimizing the divergence of the original data matrix and its non-negative approximation. Experimental results show that P-NMF derives bases which are somewhat better suitable for a localized representation than NMF.
KeywordsSingular Value Decomposition Sparse Representation Nonnegative Matrix Factorization Positive Matrix Factorization Original Data Matrix
- 5.Lee, D.D., Seung, H.S.: Algorithms for non-negative matrix factorization. In: NIPS, pp. 556–562 (2000)Google Scholar
- 8.Kawamoto, T., Hotta, K., Mishima, T., Fujiki, J., Tanaka, M., Kurita, T.: Estimation of single tones from chord sounds using non-negative matrix factorization. Neural Network World 3, 429–436 (2000)Google Scholar
- 9.Saul, L.K., Lee, D.D.: Multiplicative updates for classification by mixture modela. In: Advances in Neural Information Processing Systems, vol. 14 (2002)Google Scholar
- 10.van Hateren, J.H., van der Schaaf, A.: Independent component filters of natural images compared with simple cells in primary visual cortex. Proc. Royal Soc. London B, 265, 2315–2320 (1998)Google Scholar