Constrained Spectral Clustering via Exhaustive and Efficient Constraint Propagation

  • Zhiwu Lu
  • Horace H. S. Ip
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6316)


This paper presents an exhaustive and efficient constraint propagation approach to exploiting pairwise constraints for spectral clustering. Since traditional label propagation techniques cannot be readily generalized to propagate pairwise constraints, we tackle the constraint propagation problem inversely by decomposing it to a set of independent label propagation subproblems which are further solved in quadratic time using semi-supervised learning based on k-nearest neighbors graphs. Since this time complexity is proportional to the number of all possible pairwise constraints, our approach gives a computationally efficient solution for exhaustively propagating pairwise constraint throughout the entire dataset. The resulting exhaustive set of propagated pairwise constraints are then used to adjust the weight (or similarity) matrix for spectral clustering. It is worth noting that this paper first clearly shows how pairwise constraints are propagated independently and then accumulated into a conciliatory closed-form solution. Experimental results on real-life datasets demonstrate that our approach to constrained spectral clustering outperforms the state-of-the-art techniques.


Spectral Cluster Constraint Propagation Adjusted Rand Index Pairwise Constraint Scene Dataset 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Wagstaff, K., Cardie, C., Rogers, S., Schroedl, S.: Constrained k-means clustering with background knowledge. In: ICML, pp. 577–584 (2001)Google Scholar
  2. 2.
    Klein, D., Kamvar, S., Manning, C.: From instance-level constraints to space-level constraints: Making the most of prior knowledge in data clustering. In: ICML, pp. 307–314 (2002)Google Scholar
  3. 3.
    Basu, S., Bilenko, M., Mooney, R.: A probabilistic framework for semi-supervised clustering. In: KDD, pp. 59–68 (2004)Google Scholar
  4. 4.
    Kulis, B., Basu, S., Dhillon, I., Mooney, R.: Semi-supervised graph clustering: A kernel approach. In: ICML, pp. 457–464 (2005)Google Scholar
  5. 5.
    Lu, Z., Peng, Y.: A semi-supervised learning algorithm on Gaussian mixture with automatic model selection. Neural Processing Letters 27, 57–66 (2008)CrossRefGoogle Scholar
  6. 6.
    Ng, A., Jordan, M., Weiss, Y.: On spectral clustering: Analysis and an algorithm. In: Advances in Neural Information Processing Systems, vol. 14, pp. 849–856 (2002)Google Scholar
  7. 7.
    von Luxburg, U.: A tutorial on spectral clustering. Statistics and Computing 17, 395–416 (2007)CrossRefMathSciNetGoogle Scholar
  8. 8.
    Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEE Trans. on Pattern Analysis and Machine Intelligence 22, 888–905 (2000)CrossRefGoogle Scholar
  9. 9.
    Veksler, O.: Star shape prior for graph-cut image segmentation. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part III. LNCS, vol. 5304, pp. 454–467. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  10. 10.
    Kamvar, S., Klein, D., Manning, C.: Spectral learning. In: IJCAI, pp. 561–566 (2003)Google Scholar
  11. 11.
    Lu, Z., Carreira-Perpinan, M.: Constrained spectral clustering through affinity propagation. In: CVPR (2008)Google Scholar
  12. 12.
    Li, Z., Liu, J., Tang, X.: Pairwise constraint propagation by semidefinite programming for semi-supervised classification. In: ICML, pp. 576–583 (2008)Google Scholar
  13. 13.
    Yu, S., Shi, J.: Segmentation given partial grouping constraints. IEEE Trans. on Pattern Analysis and Machine Intelligence 26, 173–183 (2004)CrossRefGoogle Scholar
  14. 14.
    Zhou, D., Bousquet, O., Lal, T., Weston, J., Schölkopf, B.: Learning with local and global consistency. In: Advances in Neural Information Processing Systems, vol. 16, pp. 321–328 (2004)Google Scholar
  15. 15.
    Lu, Z., Ip, H.: Image categorization by learning with context and consistency. In: CVPR, pp. 2719–2726 (2009)Google Scholar
  16. 16.
    Lu, Z., Ip, H.: Combining context, consistency, and diversity cues for interactive image categorization. IEEE Transactions on Multimedia 12, 194–203 (2010)CrossRefGoogle Scholar
  17. 17.
    Law, M., Topchy, A., Jain, A.: Clustering with soft and group constraints. In: Proceedings of the Joint IAPR International Workshop on Structural, Syntactic, and Statistical Pattern Recognition, pp. 662–670 (2004)Google Scholar
  18. 18.
    Law, M., Topchy, A., Jain, A.: Model-based clustering with probabilistic constraints. In: Proceedings of SIAM Data Mining, pp. 641–645 (2005)Google Scholar
  19. 19.
    Hubert, L., Arabie, P.: Comparing partitions. Journal of Classification 2, 193–218 (1985)CrossRefGoogle Scholar
  20. 20.
    Lu, Z., Peng, Y., Xiao, J.: From comparing clusterings to combining clusterings. In: AAAI, pp. 665–670 (2008)Google Scholar
  21. 21.
    Lu, Z., Peng, Y., Ip, H.: Gaussian mixture learning via robust competitive agglomeration. Pattern Recognition Letters 31, 539–547 (2010)CrossRefGoogle Scholar
  22. 22.
    Oliva, A., Torralba, A.: Modeling the shape of the scene: A holistic representation of the spatial envelope. IJCV 42, 145–175 (2001)zbMATHCrossRefGoogle Scholar
  23. 23.
    Bosch, A., Zisserman, A., Muñoz, X.: Scene classification via pLSA. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3954, pp. 517–530. Springer, Heidelberg (2006)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Zhiwu Lu
    • 1
  • Horace H. S. Ip
    • 2
  1. 1.Department of Computer ScienceCity University of Hong KongHong Kong
  2. 2.AIMtech CentreCity University of Hong KongHong Kong

Personalised recommendations