Theory of Computing Systems

, Volume 46, Issue 3, pp 416–442 | Cite as

Small Space Representations for Metric Min-sum k-Clustering and Their Applications

Article
First Online:
Received:
Accepted:

Abstract

The min-sum k -clustering problem is to partition a metric space (P,d) into k clusters C 1,…,C k ⊆P such that \(\sum_{i=1}^{k}\sum_{p,q\in C_{i}}d(p,q)\) is minimized. We show the first efficient construction of a coreset for this problem. Our coreset construction is based on a new adaptive sampling algorithm. With our construction of coresets we obtain two main algorithmic results.

The first result is a sublinear-time (4+ε)-approximation algorithm for the min-sum k-clustering problem in metric spaces. The running time of this algorithm is \(\widetilde{{\mathcal{O}}}(n)\) for any constant k and ε, and it is o(n 2) for all k=o(log n/log log n). Since the full description size of the input is Θ(n 2), this is sublinear in the input size. The fastest previously known o(log n)-factor approximation algorithm for k>2 achieved a running time of Ω(n k ), and no non-trivial o(n 2)-time algorithm was known before.

Our second result is the first pass-efficient data streaming algorithm for min-sum k-clustering in the distance oracle model, i.e., an algorithm that uses poly(log n,k) space and makes 2 passes over the input point set, which arrives in form of a data stream in arbitrary order. It computes an implicit representation of a clustering of (P,d) with cost at most a constant factor larger than that of an optimal partition. Using one further pass, we can assign each point to its corresponding cluster.

To develop the coresets, we introduce the concept of α -preserving metric embeddings. Such an embedding satisfies properties that the distance between any pair of points does not decrease and the cost of an optimal solution for the considered problem on input (P,d′) is within a constant factor of the optimal solution on input (P,d). In other words, the goal is to find a metric embedding into a (structurally simpler) metric space that approximates the original metric up to a factor of α with respect to a given problem. We believe that this concept is an interesting generalization of coresets.

Keywords

Sublinear-time algorithms Sublinear-space algorithms Clustering Min-sum k-clustering Balanced k-median Streaming algorithms Coresets 
This is a preview of subscription content, log in to check access.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Abraham, I., Bartal, Y., Neiman, O.: Advances in metric embedding theory. In: Proc. 38th Annual ACM Symposium on Theory of Computing (STOC), pp. 271–286, 2006 Google Scholar
  2. 2.
    Alon, N., Dar, S., Parnas, M., Ron, D.: Testing of clustering. SIAM J. Discrete Math. 16(3), 393–417 (2003) MATHCrossRefMathSciNetGoogle Scholar
  3. 3.
    Agarwal, P.K., Har-Peled, S., Varadarajan, K.R.: Geometric approximation via coresets. In: Welzl, E. (ed.) Current Trends in Combinatorial and Computational Geometry. Cambridge University Press, New York (2005) Google Scholar
  4. 4.
    Bădoiu, M., Czumaj, A., Indyk, P., Sohler, C.: Facility location in sublinear time. In: Proc. 32nd Annual International Colloquium on Automata, Languages and Programming (ICALP), pp. 866–877, 2005 Google Scholar
  5. 5.
    Bădoiu, M., Har-Peled, S., Indyk, P.: Approximate clustering via core-sets. In: Proc. 34th Annual ACM Symposium on Theory of Computing (STOC), pp. 250–257, 2002 Google Scholar
  6. 6.
    Bartal, Y., Charikar, M., Raz, D.: Approximating min-sum k-clustering in metric spaces. In: Proc. 33rd Annual ACM Symposium on Theory of Computing (STOC), pp. 11–20, 2001 Google Scholar
  7. 7.
    Berkhin, P.: Survey of clustering data mining techniques. Technical Report, Accrue Software, San Jose, CA (2002) Google Scholar
  8. 8.
    Charikar, M., Chekuri, C., Feder, T., Motwani, R.: Incremental clustering and dynamic information retrieval. In: Proc. 29th Annual ACM Symposium on Theory of Computing (STOC), pp. 626–635, 1997 Google Scholar
  9. 9.
    Charikar, M., O’Callaghan, L., Panigrahy, R.: Better streaming algorithms for clustering problems. In: Proc. 35th Annual ACM Symposium on Theory of Computing (STOC), pp. 30–39, 2003 Google Scholar
  10. 10.
    Chen, K.: On k-median clustering in high dimensions. In: Proc. 17th Annual ACM-SIAM Symposium on Discrete Algorithms (SODA), pp. 1177–1185, 2006 Google Scholar
  11. 11.
    Czumaj, A., Sohler, C.: Abstract combinatorial programs and efficient property testers. SIAM J. Comput. 34(3), 580–615 (2005) MATHCrossRefMathSciNetGoogle Scholar
  12. 12.
    Czumaj, A., Sohler, C.: Sublinear-time approximation for clustering via random sampling. Random Struct. Algorithms 30(1–2), 226–256 (2007) MATHCrossRefMathSciNetGoogle Scholar
  13. 13.
    Czumaj, A., Sohler, C.: Sublinear-time algorithms. Bull. EATCS 89, 23–47 (2006) MATHMathSciNetGoogle Scholar
  14. 14.
    Fernandez de la Vega, W., Karpinski, M., Kenyon, C., Rabani, Y.: Approximation schemes for clustering problems. In: Proc. 35th Annual ACM Symposium on Theory of Computing (STOC), pp. 50–58, 2003 Google Scholar
  15. 15.
    Fernandez de la Vega, W., Kenyon, C.: A randomized approximation scheme for metric MAX-CUT. J. Comput. Syst. Sci. 63(4), 531–541 (2001) MATHCrossRefMathSciNetGoogle Scholar
  16. 16.
    Fotakis, D.: Memoryless facility location in one pass. In: Proc. 23rd Annual Symposium on Theoretical Aspects of Computer Science (STACS), pp. 608–620, 2006 Google Scholar
  17. 17.
    Frahling, G., Sohler, C.: Coresets in dynamic geometric data streams. In: Proc. 37th Annual ACM Symposium on Theory of Computing (STOC), pp. 209–217, 2005 Google Scholar
  18. 18.
    Goemans, M.X., Williamson, D.P.: Improved approximation algorithms for Maximum Cut and satisfiability problems using semidefinite programming. J. ACM 42, 1115–1145 (1995) MATHCrossRefMathSciNetGoogle Scholar
  19. 19.
    Goldreich, O., Goldwasser, S., Ron, D.: Property testing and its connection to learning and approximation. J. ACM 45(4), 653–750 (1998) MATHCrossRefMathSciNetGoogle Scholar
  20. 20.
    Guha, S., Mishra, N., Motwani, R., O’Callaghan, L.: Clustering data streams. In: Proc. 41st IEEE Symposium on Foundations of Computer Science (FOCS), pp. 359–366, 2000 Google Scholar
  21. 21.
    Gutmann-Beck, N., Hassin, R.: Approximation algorithms for min-sum p-clustering. Discrete Appl. Math. 89, 125–142 (1998) CrossRefMathSciNetGoogle Scholar
  22. 22.
    Har-Peled, S.: Clustering motion. In: Proc. 42nd IEEE Symposium on Foundations of Computer Science (FOCS), pp. 84–93, 2001 Google Scholar
  23. 23.
    Har-Peled, S., Mazumdar, S.: Coresets for k-means and k-medians and their applications. In: Proc. 36th Annual ACM Symposium on Theory of Computing (STOC), pp. 291–300, 2004 Google Scholar
  24. 24.
    Har-Peled, S., Kushal, A.: Smaller coresets for k-median and k-means clustering. In: Proc. 21st Annual ACM Symposium on Computational Geometry (SoCG), pp. 126–134, 2005 Google Scholar
  25. 25.
    Har-Peled, S., Varadarajan, K.: Projective clustering in high dimensions using core-sets. In: Proc. 18th Annual ACM Symposium on Computational Geometry (SoCG), pp. 312–318, 2002 Google Scholar
  26. 26.
    Indyk, P.: Sublinear time algorithms for metric space problems. In: Proc. 31st Annual ACM Symposium on Theory of Computing (STOC), pp. 428–434, 1999 Google Scholar
  27. 27.
    Indyk, P.: High-dimensional computational geometry. PhD thesis, Stanford University (2000) Google Scholar
  28. 28.
    Indyk, P.: Algorithms for dynamic geometric problems over data streams. In: Proc. 36th Annual ACM Symposium on Theory of Computing (STOC), pp. 373–380, 2004 Google Scholar
  29. 29.
    Jain, A.K., Murty, M.N., Flynn, P.J.: Data clustering: a review. ACM Comput. Surv. 31(3), 264–323 (2003) CrossRefGoogle Scholar
  30. 30.
    Korn, F., Muthukrishnan, S., Srivastava, D.: Reverse nearest neighbor aggregates over data streams. In: Proc. 28th International Conference on Very Large Data Bases (VLDB), pp. 814–825, 2002 Google Scholar
  31. 31.
    Kumar, A., Sabharwal, Y., Sen, S.: A simple linear time (1+ε)-approximation algorithm for k-means clustering in any dimensions. In: Proc. 45th IEEE Symposium on Foundations of Computer Science (FOCS), pp. 454–462, 2004 Google Scholar
  32. 32.
    Kumar, A., Sabharwal, Y., Sen, S.: Linear time algorithms for clustering problems in any dimensions. In: Proc. 32nd Annual International Colloquium on Automata, Languages and Programming (ICALP), pp. 1374–1385, 2005 Google Scholar
  33. 33.
    Mettu, R., Plaxton, G.: Optimal time bounds for approximate clustering. Machine Learning 56(1–3), 35–60 (2004) MATHCrossRefGoogle Scholar
  34. 34.
    Meyerson, A.: Online facility location. In: Proc. 42nd IEEE Symposium on Foundations of Computer Science (FOCS), pp. 426–431, 2001 Google Scholar
  35. 35.
    Meyerson, A., O’Callaghan, L., Plotkin, S.: A k-median algorithm with running time independent of data size. Machine Learning 56(1–3), 61–87 (2004) MATHCrossRefGoogle Scholar
  36. 36.
    Mishra, N., Oblinger, D., Pitt, L.: Sublinear time approximate clustering. In: Proc. 12th Annual ACM-SIAM Symposium on Discrete Algorithms (SODA), pp. 439–447, 2001 Google Scholar
  37. 37.
    Muthukrishnan, S.: Data streams: algorithms and applications. Found. Trends Theor. Comput. Sci. 1(2) (2005) Google Scholar
  38. 38.
    Parnas, M., Ron, D., Rubinfeld, R.: Tolerant property testing and distance approximation. Electronic Colloquium on Computational Complexity (ECCC), Report No. 10 (2004) Google Scholar
  39. 39.
    Rabinovich, Y.: On average distortion of embedding metrics into the line and into L 1. In: Proc. 35th Annual ACM Symposium on Theory of Computing (STOC), pp. 456–462, 2003 Google Scholar
  40. 40.
    Sahni, S., Gonzalez, T.: P-complete approximation problems. J. ACM 23, 555–566 (1976) MATHCrossRefMathSciNetGoogle Scholar
  41. 41.
    Schulman, L.J.: Clustering for edge-cost minimization. In: Proc. 32nd Annual ACM Symposium on Theory of Computing (STOC), pp. 547–555, 2000 Google Scholar
  42. 42.
    Thorup, M.: Quick k-median, k-center, and facility location for sparse graphs. SIAM J. Comput. 34(2), 405–432 (2005) MATHCrossRefMathSciNetGoogle Scholar
  43. 43.
    Tokuyama, T., Nakano, J.: Geometric algorithms for the minimum cost assignment problem. Random Struct. Algorithms 6(4), 393–406 (1995) MATHCrossRefMathSciNetGoogle Scholar
  44. 44.
    Xu, R., II Wunsch, D.: Survey of clustering algorithms. IEEE Trans. Neural Netw. 16(3), 645–678 (2005) CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2009

Authors and Affiliations

  1. 1.Department of Computer Science and Centre for Discrete Mathematics and its ApplicationsUniversity of WarwickCoventryUK
  2. 2.Department of Computer ScienceTU DortmundDortmundGermany

Personalised recommendations