Machine Learning

, Volume 56, Issue 1, pp 9–33

Clustering Large Graphs via the Singular Value Decomposition

  • P. Drineas
  • A. Frieze
  • R. Kannan
  • S. Vempala
  • V. Vinay

DOI: 10.1023/B:MACH.0000033113.59016.96

Cite this article as:
Drineas, P., Frieze, A., Kannan, R. et al. Machine Learning (2004) 56: 9. doi:10.1023/B:MACH.0000033113.59016.96


We consider the problem of partitioning a set of m points in the n-dimensional Euclidean space into k clusters (usually m and n are variable, while k is fixed), so as to minimize the sum of squared distances between each point and its cluster center. This formulation is usually the objective of the k-means clustering algorithm (Kanungo et al. (2000)). We prove that this problem in NP-hard even for k = 2, and we consider a continuous relaxation of this discrete problem: find the k-dimensional subspace V that minimizes the sum of squared distances to V of the m points. This relaxation can be solved by computing the Singular Value Decomposition (SVD) of the m × n matrix A that represents the m points; this solution can be used to get a 2-approximation algorithm for the original problem. We then argue that in fact the relaxation provides a generalized clustering which is useful in its own right.

Finally, we show that the SVD of a random submatrix—chosen according to a suitable probability distribution—of a given matrix provides an approximation to the SVD of the whole matrix, thus yielding a very fast randomized algorithm. We expect this algorithm to be the main contribution of this paper, since it can be applied to problems of very large size which typically arise in modern applications.

Singular Value Decomposition randomized algorithms k-means clustering 
Download to read the full article text

Copyright information

© Kluwer Academic Publishers 2004

Authors and Affiliations

  • P. Drineas
    • 1
  • A. Frieze
    • 2
  • R. Kannan
    • 3
  • S. Vempala
    • 4
  • V. Vinay
    • 5
  1. 1.Computer Science DepartmentRensselaer Polytechnic InstituteTroy
  2. 2.Department of Mathematical SciencesCarnegie Mellon UniversityPittsburgh
  3. 3.Computer Science DepartmentYale UniversityNew Haven
  4. 4.Department of MathematicsM.I.T.Cambridge
  5. 5.Indian Institute of ScienceBangaloreIndia

Personalised recommendations