Nonnegative Lagrangian Relaxation of K-Means and Spectral Clustering

  • Chris Ding
  • Xiaofeng He
  • Horst D. Simon
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3720)


We show that K-means and spectral clustering objective functions can be written as a trace of quadratic forms. Instead of relaxation by eigenvectors, we propose a novel relaxation maintaining the nonnegativity of the cluster indicators and thus give the cluster posterior probabilities, therefore resolving cluster assignment difficulty in spectral relaxation. We derive a multiplicative updating algorithm to solve the nonnegative relaxation problem. The method is briefly extended to semi-supervised classification and semi-supervised clustering.


Bipartite Graph Spectral Cluster Nonnegative Matrix Factorization Spectral Relaxation Simultaneous Cluster 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Bach, F.R., Jordan, M.I.: Learning spectral clustering. In: Neural Info. Processing Systems 16, NIPS 2003 (2003)Google Scholar
  2. 2.
    Ding, C., He, X., Simon, H.D.: On the equivalence of nonnegative matrix factorization and spectral clustering. In: Proc. SIAM Data Mining Conf. (2005)Google Scholar
  3. 3.
    Lee, D.D., Seung, H.S.: Algorithms for non-negatvie matrix factorization. In: Dietterich, T.G., Tresp, V. (eds.) Advances in Neural Information Processing Systems, vol. 13. The MIT Press, Cambridge (2001)Google Scholar
  4. 4.
    Sha, F., Saul, L.K., Lee, D.D.: Multiplicative updates for nonnegative quadratic programming in support vector machines. In: Advances in Neural Information Processing Systems 15, pp. 1041–1048 (2003)Google Scholar
  5. 5.
    Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEE Trans. on Pattern Analysis and Machine Intelligence 22, 888–905 (2000)CrossRefGoogle Scholar
  6. 6.
    Wagstaff, K., Cardie, C., Rogers, S., Schroedl, S.: Constrained k-means clustering with background knowledge. In: Proc. Int’l. Conf. Machine Learning (2001)Google Scholar
  7. 7.
    Zha, H., He, X., Ding, C., Gu, M., Simon, H.D.: Bipartite graph partitioning and data clustering. In: Proc. Int’l. Conf. Information and Knowledge Management, CIKM 2001 (2001)Google Scholar
  8. 8.
    Zhou, D., Bousquet, O., Lal, T.N., Weston, J., Schölkopf, B.: Learning with local and global consistency. In: Proc. Neural Info. Processing Systems (2003)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Chris Ding
    • 1
  • Xiaofeng He
    • 1
  • Horst D. Simon
    • 1
  1. 1.Lawrence Berkeley National LaboratoryBerkeleyUSA.

Personalised recommendations