Abstract
Nonnegative matrix factorization (NMF) is an unsupervised learning method for decomposing high-dimensional nonnegative data matrices and extracting basic and intrinsic features. Since image data are described and stored as nonnegative matrices, the mining and analysis process usually involves the use of various NMF strategies. NMF methods have well-known applications in face recognition, image reconstruction, handwritten digit recognition, image denoising and feature extraction. Recently, several projective NMF (P-NMF) methods based on positively constrained projections have been proposed and were found to perform better than the standard NMF approach in some aspects. However, some drawbacks still affect the existing NMF and P-NMF algorithms; these include dense factors, slow convergence, learning poor local features, and low reconstruction accuracy. The aim of this paper is to design algorithms that address the aforementioned issues. In particular, we propose two embedded P-NMF algorithms: the first method combines the alternating least squares (ALS) algorithm with the P-NMF update rules of the Frobenius norm and the second one embeds ALS with the P-NMF update rule of the Kullback–Leibler divergence. To assess the performances of the proposed methods, we conducted various experiments on four well-known data sets of faces. The experimental results reveal that the proposed algorithms outperform other related methods by providing very sparse factors and extracting better localized features. In addition, the empirical studies show that the new methods provide highly orthogonal factors that possess small entropy values.
Similar content being viewed by others
Notes
Due to the nonnegativity constraints in NMF, its factors can never be completely orthogonal. Therefore, in this paper, by orthogonality (“orthogonal”) we mean approximate orthogonality.
References
Belhumeur P, Hespanha J, Kriegman D (1997) Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Trans Pattern Anal Mach Intell 19(7):711–720
Berry MW, Browne M, Langville AN, Pauca VP, Plemmons RJ (2007) Algorithms and applications for approximate nonnegative matrix factorization. Comput Stat Data Anal 52:155–173
Ding C, Li T, Peng W, Park H (2006) Orthogonal nonnegative matrix tri-factorizations for clustering. In: Proceedings of the 12th ACM SIGKDD international conference on knowledge discovery and data mining KDD06, pp 126–135
Daube-Witherspoon ME, Muehllehner G (1986) An iterative image space reconstruction algorithm suitable for volume ECT. IEEE Trans Med Imaging MI 5(2):61–66
Hoyer PO (2004) Nonnegative matrix factorization with sparseness constraints. J Mach Learn Res 5:1457–1469
Hu L, Dai L, Wu J (2014) Convergent projective non-negative matrix factorization with Kullback–Leibler divergence. Pattern Recognit Lett 36:15–21
Hu L, Wu J, Wang L (2013) Convergent projective nonnegative matrix factorization. IJCSI Int J Comput Sci Issues 10(1):127–133
Hu L, Wu J, Wang L (2013) Linear projective non-negative matrix factorization. Res J Appl Sci Eng Technol 6(9):1626–1631
Lee DD, Seung HS (1999) Learning the parts of objects by non-negative matrix factorization. Nature 401:788–791
Lee DD, Seung HS (2001) Algorithms for non-negative matrix factorization. Adv Neural Inf Process Syst 13:556–562
Li SZ, Hou XW, Zhang HJ, Cheng QS (2001) Learning spatially localized, parts-based representation. IEEE Comput Vis Pattern Recognit 1:507–512
Long X, Lu H, Peng Y (2013) Sparse non-negative matrix factorization based on spatial pyramid matching for face recognition. In: IEEE fifth international conference on intelligent human-machine systems and cybernetics IHMSC, pp 1828–1832
Nefian AV (1999) A hidden Markov model based approach for face detection and recognition. Ph.D. thesis, Georgia Institute of Technology, Atlanta
Oja E, Yang Z (2010) Orthogonal nonnegative learning for sparse feature extraction and approximate combinatorial optimization. Front Electr Electron Eng China 5(3):261–273
Pauca VP, Piper J, Plemmons RJ (2006) Nonnegative matrix factorization for spectral data analysis. Linear Algebra Appl 416:29–47
Pompili F, Gillis N, Absil P-A, Glineur F (2014) Two algorithms for orthogonal nonnegative matrix factorization with application to clustering. Neurocomputing 141:15–25
Samaria F, Harter A (1994) Parameterization of a stochastic model for human face identification. In Proceedings of the 2nd IEEE workshop on applications of computer vision, pp 138–1142
Shahnaz F, Berry MW, Pauca P, Plemmons RJ (2006) Document clustering using nonnegative matrix factorization. Inf Process Manag 42(2):373–386
Sung K. K. (1996) Learning and Example Selection for object and pattern recognition. Ph.D. thesis, MIT, Artificial Intelligence Laboratory and Center for Biological and Computational Learning, Cambridge
Wang N, Du B, Zhang L (2013) An endmember dissimilarity constrained non-negative matrix factorization method for hyperspectral unmixing. IEEE J Sel Topics Appl Earth Obs Remote Sens 6(2):554–569
Yang Z, Oja E (2010) Linear and nonlinear projective nonnegative matrix factorization. IEEE Trans Neural Netw 21(5):734–749
Yang Z, Yuan Z, Laaksonen J (2007) Projective nonnegative matrix factorization with applications to facial image processing. Int J Pattern Recognit Artif Intell 21(8):1353–1362
Yuan Z (2009) Advances in independent component analysis and nonnegative matrix factorization. Ph.D. thesis, Helsinki University of Technology, Dissertations in Information and Computer Sciene
Yuan Z, Oja E (2005) Projective nonnegative matrix factorization for image compression and feature extraction. In Proceedings of the 14th Scandinavian conference on image analysis, Springer lecture notes in computer science, vol 3540, pp 333–342
Yuan Z, Oja E (2007) A family of modified projective nonnegative matrix factorization algorithms. IEEE: Proceedings of the 9th international symposium on signal processing and its applications, Sharjah, United Arab Emirates, pp 1–4
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Belachew, M.T., Del Buono, N. Robust embedded projective nonnegative matrix factorization for image analysis and feature extraction. Pattern Anal Applic 20, 1045–1060 (2017). https://doi.org/10.1007/s10044-016-0545-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10044-016-0545-z
Keywords
- Embedded projective nonnegative matrix factorization
- Orthogonality
- Sparsity
- Entropy
- Reconstruction accuracy
- Feature extraction