Orthogonal Nonnegative Matrix Factorization: Multiplicative Updates on Stiefel Manifolds

  • Jiho Yoo
  • Seungjin Choi
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5326)


Nonnegative matrix factorization (NMF) is a popular method for multivariate analysis of nonnegative data, the goal of which is decompose a data matrix into a product of two factor matrices with all entries in factor matrices restricted to be nonnegative. NMF was shown to be useful in a task of clustering (especially document clustering). In this paper we present an algorithm for orthogonal nonnegative matrix factorization, where an orthogonality constraint is imposed on the nonnegative decomposition of a term-document matrix. We develop multiplicative updates directly from true gradient on Stiefel manifold, whereas existing algorithms consider additive orthogonality constraints. Experiments on several different document data sets show our orthogonal NMF algorithms perform better in a task of clustering, compared to the standard NMF and an existing orthogonal NMF.


Nonnegative Matrix Factorization Document Cluster Orthogonality Constraint Multiplicative Update Probabilistic Latent Semantic Indexing 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Lee, D.D., Seung, H.S.: Learning the parts of objects by non-negative matrix factorization. Nature 401, 788–791 (1999)CrossRefzbMATHGoogle Scholar
  2. 2.
    Li, S.Z., Hou, X.W., Zhang, H.J., Cheng, Q.S.: Learning spatially localized parts-based representation. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), Kauai, Hawaii, pp. 207–212 (2001)Google Scholar
  3. 3.
    Ding, C., Li, T., Peng, W., Park, H.: Orthogonal nonnegative matrix tri-factorizations for clustering. In: Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), Philadelphia, PA (2006)Google Scholar
  4. 4.
    Xu, W., Liu, X., Gong, Y.: Document clustering based on non-negative matrix factorization. In: Proceedings of the ACM SIGIR International Conference on Research and Development in Information Retrieval (SIGIR), Toronto, Canada (2003)Google Scholar
  5. 5.
    Shahnaz, F., Berry, M., Pauca, P., Plemmons, R.: Document clustering using nonnegative matrix factorization. Information Processing and Management 42, 373–386 (2006)CrossRefzbMATHGoogle Scholar
  6. 6.
    Lee, D.D., Seung, H.S.: Algorithms for non-negative matrix factorization. In: Advances in Neural Information Processing Systems (NIPS), vol. 13. MIT Press, Cambridge (2001)Google Scholar
  7. 7.
    Gaussier, E., Goutte, C.: Relation between PLSA and NMF and implications. In: Proceedings of the ACM SIGIR International Conference on Research and Development in Information Retrieval (SIGIR), Salvador, Brazil (2005)Google Scholar
  8. 8.
    Choi, S.: Algorithms for orthogonal nonnegative matrix factorization. In: Proceedings of the International Joint Conference on Neural Networks (IJCNN), Hong Kong (2008)Google Scholar
  9. 9.
    Ding, C., He, X., Simon, H.D.: On the equivalence of nonnegative matrix factorization and spectral clustering. In: Proceedings of the SIAM International Conference on Data Mining (SDM), Newport Beach, CA, pp. 606–610 (2005)Google Scholar
  10. 10.
    Smith, S.T.: Geometric Optimization Methods for Adaptive Filtering. Ph.D thesis, Harvard University (1993)Google Scholar
  11. 11.
    Edelman, A., Arias, T., Smith, S.T.: The geometry of algorithms with orthogonality constraints. SIAM J. Matrix Anal. Appl. 20(2), 303–353 (1998)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Stiefel, E.: Richtungsfelder und fernparallelismus in n-dimensionalem mannig faltigkeiten. Commentarii Math. Helvetici 8, 305–353 (1935)MathSciNetCrossRefGoogle Scholar
  13. 13.
    Lovasz, L., Plummer, M.: Matching Theory. Akademiai Kiado (1986)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Jiho Yoo
    • 1
  • Seungjin Choi
    • 1
  1. 1.Department of Computer SciencePohang University of Science and TechnologyPohangKorea

Personalised recommendations