Regularized NNLS Algorithms for Nonnegative Matrix Factorization with Application to Text Document Clustering
Nonnegative Matrix Factorization (NMF) has recently received much attention both in an algorithmic aspect as well as in applications. Text document clustering and supervised classification are important applications of NMF. Various types of numerical optimization algorithms have been proposed for NMF, which includes multiplicative, projected gradient descent, alternating least squares and active-set ones. In this paper, we discuss the selected Non-Negatively constrained Least Squares (NNLS) algorithms (a family of the NNLS algorithm proposed by Lawson and Hanson) that belong to a class of active-set methods. We noticed that applying the NNLS algorithm to the Tikhonov regularized LS objective function with a regularization parameter exponentially decreasing considerably increases the accuracy of data clustering as well as it reduces the risk of getting stuck into unfavorable local minima. Moreover, the experiments demonstrate that the regularized NNLS algorithm is superior to many well-known NMF algorithms used for text document clustering.
KeywordsMonte Carlo Nonnegative Matrix Factorization Nonnegative Matrix Document Cluster Latent Semantic Indexing
Unable to display preview. Download preview PDF.
- 4.Buciu, I., Pitas, I.: Application of non-negative and local nonnegative matrix factorization to facial expression recognition. In: Proc. Intl. Conf. Pattern Recognition (ICPR), pp. 288–291 (2004)Google Scholar
- 5.Cai, D., He, X., Wu, X., Bao, H., Han, J.: Locality preserving nonnegative matrix factorization. In: Proc. IJCAI 2009, pp. 1010–1015 (2009)Google Scholar
- 6.Cai, D., He, X., Wu, X., Han, J.: Nonnegative matrix factorization on manifold. In: Proc. 8th IEEE Intl. Conf. Data Mining (ICDM), pp. 63–72 (2008)Google Scholar
- 7.Cichocki, A., Zdunek, R., Phan, A.H., Amari, S.I.: Nonnegative Matrix and Tensor Factorizations: Applications to Exploratory Multi-way Data Analysis and Blind Source Separation. Wiley and Sons, Chichester (2009)Google Scholar
- 9.Ding, C., Li, T., Peng, W.: Nonnegative matrix factorization and probabilistic latent semantic indexing: Equivalence, chi-square statistic, and a hybrid method. In: Proc. AAAI National Conf. Artificial Intelligence (AAAI 2006) (2006)Google Scholar
- 14.Jankowiak, M.: Application of nonnegative matrix factorization for text document classification. MSc thesis (supervised by Dr. R. Zdunek), Wroclaw University of Technology, Poland (2010) (in Polish)Google Scholar
- 23.Sra, S., Dhillon, I.S.: Nonnegative matrix approximation: Algorithms and Applications. UTCS Technical Report TR-06-27, Austin, USA (2006), http://www.cs.utexas.edu/ftp/pub/techreports/tr06-27.pdf
- 25.Zdunek, R., Cichocki, A.: Comput. Intel. Neurosci. (939567) (2008)Google Scholar
- 26.Zdunek, R., Phan, A.H., Cichocki, A.: Aust. J. Intel. Inform. Process. Syst. 12, 16–22 (2010)Google Scholar