Large Scale Spectral Clustering Using Resistance Distance and Spielman-Teng Solvers

  • Nguyen Lu Dang Khoa
  • Sanjay Chawla
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7569)

Abstract

The promise of spectral clustering is that it can help detect complex shapes and intrinsic manifold structure in large and high dimensional spaces. The price for this promise is the computational cost O(n 3) for computing the eigen-decomposition of the graph Laplacian matrix - so far a necessary subroutine for spectral clustering. In this paper we bypass the eigen-decomposition of the original Laplacian matrix by leveraging the recently introduced Spielman and Teng near-linear time solver for systems of linear equations and random projection. Experiments on several synthetic and real datasets show that the proposed approach has better clustering quality and is faster than the state-of-the-art approximate spectral clustering methods.

Keywords

spectral clustering resistance distance Spielman-Teng Solver random projection 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Achlioptas, D.: Database-friendly random projections. In: Proceedings of the 20th ACM Symposium on Principles of Database Systems, pp. 274–281. ACM, New York (2001)Google Scholar
  2. 2.
    Chen, W.Y., Song, Y., Bai, H., Lin, C.J., Chang, E.Y.: Parallel spectral clustering in distributed systems. IEEE TPAMI 33(3), 568–586 (2011)CrossRefGoogle Scholar
  3. 3.
    Chen, X., Cai, D.: Large scale spectral clustering with landmark-based representation. In: AAAI (2011)Google Scholar
  4. 4.
    Doyle, P.G., Snell, J.L.: Random Walks and Electric Networks. Mathematical Association of America, Washington, DC (1984)MATHGoogle Scholar
  5. 5.
    Fouss, F., Renders, J.M.: Random-walk computation of similarities between nodes of a graph with application to collaborative recommendation. IEEE TKDE 19(3), 355–369 (2007)Google Scholar
  6. 6.
    Fowlkes, C., Belongie, S., Chung, F., Malik, J.: Spectral grouping using the nyström method. IEEE TPAMI 26, 214–225 (2004)CrossRefGoogle Scholar
  7. 7.
    Frank, A., Asuncion, A.: UCI machine learning repository (2010)Google Scholar
  8. 8.
    Johnson, W., Lindenstrauss, J.: Extensions of Lipschitz mappings into a Hilbert space. In: Conference in Modern Analysis and Probability, Contemporary Mathematics, vol. 26, pp. 189–206. American Mathematical Society (1984)Google Scholar
  9. 9.
    Koutis, I., Miller, G.L., Tolliver, D.: Combinatorial Preconditioners and Multilevel Solvers for Problems in Computer Vision and Image Processing. In: Bebis, G., Boyle, R., Parvin, B., Koracin, D., Kuno, Y., Wang, J., Wang, J.-X., Wang, J., Pajarola, R., Lindstrom, P., Hinkenjann, A., Encarnação, M.L., Silva, C.T., Coming, D. (eds.) ISVC 2009, Part I. LNCS, vol. 5875, pp. 1067–1078. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  10. 10.
    Luxburg, U.: A tutorial on spectral clustering. Statistics and Computing 17(4) (2007)Google Scholar
  11. 11.
    von Luxburg, U., Radl, A., Hein, M.: Getting lost in space: Large sample analysis of the resistance distance. In: NIPS, pp. 2622–2630 (2010)Google Scholar
  12. 12.
    von Luxburg, U., Bousquet, O., Belkin, M.: On the Convergence of Spectral Clustering on Random Samples: The Normalized Case. In: Shawe-Taylor, J., Singer, Y. (eds.) COLT 2004. LNCS (LNAI), vol. 3120, pp. 457–471. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  13. 13.
    Mavroeidis, D.: Accelerating spectral clustering with partial supervision. Data Min. Knowl. Discov. 21, 241–258 (2010)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Ng, A.Y., Jordan, M.I., Weiss, Y.: On spectral clustering: Analysis and an algorithm. In: Advances in Neural Information Processing Systems, pp. 849–856. MIT Press (2001)Google Scholar
  15. 15.
    Qiu, H., Hancock, E.: Clustering and embedding using commute times. IEEE TPAMI 29(11), 1873–1890 (2007)CrossRefGoogle Scholar
  16. 16.
    Spielman, D.A., Srivastava, N.: Graph sparsification by effective resistances. In: Proceedings of the 40th Annual ACM STOC, pp. 563–568. ACM (2008)Google Scholar
  17. 17.
    Spielman, D.A., Teng, S.H.: Nearly-linear time algorithms for graph partitioning, graph sparsification, and solving linear systems. In: Proceedings of the 36th Annual ACM STOC, pp. 81–90. ACM, New York (2004)Google Scholar
  18. 18.
    Spielman, D.A., Teng, S.H.: Nearly-linear time algorithms for preconditioning and solving symmetric, diagonally dominant linear systems. CoRR abs/cs/0607105 (2006)Google Scholar
  19. 19.
    Venkatasubramanian, S., Wang, Q.: The johnson-lindenstrauss transform: An empirical study. In: ALENEX 2011 (2011)Google Scholar
  20. 20.
    Wang, L., Leckie, C., Ramamohanarao, K., Bezdek, J.: Approximate Spectral Clustering. In: Theeramunkong, T., Kijsirikul, B., Cercone, N., Ho, T.-B. (eds.) PAKDD 2009. LNCS, vol. 5476, pp. 134–146. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  21. 21.
    Yan, D., Huang, L., Jordan, M.I.: Fast approximate spectral clustering. In: Proceedings of the 15th ACM SIGKDD, pp. 907–916 (2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Nguyen Lu Dang Khoa
    • 1
  • Sanjay Chawla
    • 1
  1. 1.School of ITUniversity of SydneyAustralia

Personalised recommendations