Journal of Intelligent Information Systems

, Volume 44, Issue 2, pp 289–308 | Cite as

A scalable approach to spectral clustering with SDD solvers

Article

Abstract

The promise of spectral clustering is that it can help detect complex shapes and intrinsic manifold structure in large and high dimensional spaces. The price for this promise is the expensive computational cost for computing the eigen-decomposition of the graph Laplacian matrix—so far a necessary subroutine for spectral clustering. In this paper we bypass the eigen-decomposition of the original Laplacian matrix by leveraging the recently introduced near-linear time solver for symmetric diagonally dominant (SDD) linear systems and random projection. Experiments on several synthetic and real datasets show that the proposed approach has better clustering quality and is faster than the state-of-the-art approximate spectral clustering methods.

Keywords

Spectral clustering Resistance distance SDD solver Random projection 

References

  1. Achlioptas, D. (2001). Database-friendly random projections. In Proceedings of the 20th ACM SIGMOD-SIGACT-SIGART symposium on Principles of Database Systems, PODS ’01 (pp. 274–281). New York: ACM.CrossRefGoogle Scholar
  2. Chen, X., & Cai, D. (2011). Large scale spectral clustering with landmark-based representation. In Twenty-Fifth AAAI Conference on Artificial Intelligence (pp. 313–318).Google Scholar
  3. Chen, W.Y., Song, Y., Bai, H., Lin, C.J., Chang, E.Y. (2011). Parallel spectral clustering in distributed systems. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(3), 568–586.CrossRefGoogle Scholar
  4. de Vries, T., Chawla, S., Houle, M.E. (2012). Density-preserving projections for large-scale local anomaly detection. Knowledge and Information Systems, 32(1), 25–52.CrossRefGoogle Scholar
  5. Donath, W.E., & Hoffman, A.J. (1973). Lower bounds for the partitioning of graphs. IBM Journal of Research and Development, 17, 420–425.CrossRefMATHMathSciNetGoogle Scholar
  6. Doyle, P.G., & Snell, J.L. (1984). Random walks and electric networks. Washington, DC: Mathematical Association of America.MATHGoogle Scholar
  7. Fiedler, M. (1973). Algebraic connectivity of graphs. Czechoslovak Mathematical Journal, 23, 298–305.MathSciNetGoogle Scholar
  8. Fouss, F., & Renders, J.M. (2007). Random-walk computation of similarities between nodes of a graph with application to collaborative recommendation. IEEE Transaction on Knowledge and Data Engineering, 19(3), 355–369.CrossRefGoogle Scholar
  9. Fowlkes, C., Belongie, S., Chung, F., Malik, J. (2004). Spectral grouping using the Nyström method. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26, 214–225.CrossRefGoogle Scholar
  10. Frank, A., & Asuncion, A. (2010). UCI machine learning repository. URL: http://archive.ics.uci.edu/ml. Accessed 31 Jan 2013
  11. Golub, G.H., & Van Loan, C.F. (1996). Matrix computations (3rd edn.). Baltimore: Johns Hopkins University Press.MATHGoogle Scholar
  12. Johnson, W., & Lindenstrauss, J. (1984). Extensions of Lipschitz mappings into a Hilbert space. In Conference in modern analysis and probability (New Haven, Conn., 1982), Contemporary Mathematics (Vol. 26, pp. 189–206). American Mathematical Society.Google Scholar
  13. Jolliffe, I.T. (2002). Principal component analysis (2nd edn.). Springer.Google Scholar
  14. Koutis, I., Miller, G.L., Tolliver, D. (2009). Combinatorial preconditioners and multilevel solvers for problems in computer vision and image processing. In Proceedings of the 5th international symposium on advances in visual computing: Part I, ISVC ’09 (pp. 1067–1078). Berlin, Heidelberg: Springer.Google Scholar
  15. Koutis, I., Miller, G., Peng, R. (2010). Approaching optimality for solving sdd linear systems. In 2010 51st annual IEEE symposium on Foundations of Computer Science (FOCS) (pp. 235–244).Google Scholar
  16. Koutis, I., Miller, G.L., Peng, R. (2011). A nearly-m log n time solver for sdd linear systems. In Proceedings of the 2011 IEEE 52nd annual symposium on Foundations of Computer Science, FOCS ’11 (pp. 590–598). Washington, DC: IEEE Computer Society.Google Scholar
  17. Luxburg, U. (2007). A tutorial on spectral clustering. Statistics and Computing, 17(4), 395–416.CrossRefMathSciNetGoogle Scholar
  18. Luxburg, U.V., Bousquet, O., Belkin, M. (2004). On the convergence of spectral clustering on random samples: The normalized case. In Proceedings of the 17th annual Conference on Learning Theory (COLT) (pp. 457–471). Springer.Google Scholar
  19. von Luxburg, U., Radl, A., Hein, M. (2010). Getting lost in space: Large sample analysis of the resistance distance. In NIPS (pp. 2622–2630).Google Scholar
  20. Mavroeidis, D. (2010). Accelerating spectral clustering with partial supervision. Data Mining and Knowledge Discovery, 21, 241–258.CrossRefMathSciNetGoogle Scholar
  21. Ng, A.Y., Jordan, M.I., Weiss, Y. (2001). On spectral clustering: Analysis and an algorithm. In Advances in neural information processing systems (pp. 849–856). MIT Press.Google Scholar
  22. Qiu, H., & Hancock, E. (2007). Clustering and embedding using commute times. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(11), 1873–1890.CrossRefGoogle Scholar
  23. Shi, J., & Malik, J. (2000). Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22, 888–905.CrossRefGoogle Scholar
  24. Spielman, D.A., & Srivastava, N. (2008). Graph sparsification by effective resistances. In Proceedings of the 40th annual ACM Symposium on Theory of Computing, STOC ’08 (pp. 563–568). New York: ACM.Google Scholar
  25. Spielman, D.A., & Teng, S.H. (2004). Nearly-linear time algorithms for graph partitioning, graph sparsification, and solving linear systems. In Proceedings of the 36th annual ACM Symposium on Theory of Computing, STOC ’04 (pp. 81–90). New York: ACM.Google Scholar
  26. Spielman, D.A., & Teng, S.H. (2006). Nearly-linear time algorithms for preconditioning and solving symmetric, diagonally dominant linear systems. CoRR abs/cs/0607105.Google Scholar
  27. Vaidya, P. (1991). Solving linear equations with symmetric diagonally dominant matrices by constructing good preconditioners. A Talk Based on this Manuscript was Presented at the IMA Workshop on Graph Theory and Sparse Matrix Computation, October 1991, Minneapolis.Google Scholar
  28. Venkatasubramanian, S., & Wang, Q. (2011). The Johnson–Lindenstrauss transform: An empirical study. In M. Müller-Hannemann, R.F.F. Werneck (Eds.), ALENEX (pp. 164–173). SIAM.Google Scholar
  29. Wang, L., Leckie, C., Ramamohanarao, K., Bezdek, J. (2009). Approximate spectral clustering. In Proceedings of the 13th Pacific-Asia conference on advances in knowledge discovery and data mining, PAKDD ’09 (pp. 134–146). Berlin Heidelberg: Springer.Google Scholar
  30. Yan, D., Huang, L., Jordan, M.I. (2009). Fast approximate spectral clustering. In Proceedings of the 15th ACM SIGKDD international conference on Knowledge Discovery and Data mining, KDD ’09 (pp. 907–916). New York: ACM.Google Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  1. 1.National ICT Australia (NICTA)SydneyAustralia
  2. 2.School of ITUniversity of SydneySydneyAustralia

Personalised recommendations