Skip to main content
SpringerLink
Log in
SpringerLink
Log in

A Data-Clustering Algorithm on Distributed Memory Multiprocessors

  • Conference paper
  • First Online:
Large-Scale Parallel Data Mining

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 1759))

Abstract

To cluster increasingly massive data sets that are common today in data and text mining, we propose a parallel implementation of the k-means clustering algorithm based on the message passing model. The proposed algorithm exploits the inherent data-parallelism in the k-means algorithm. We analytically show that the speedup and the scaleup of our algorithm approach the optimal as the number of data points increases. We implemented our algorithm on an IBM POWERparallel SP2 with a maximum of 16 nodes. On typical test data sets, we observe nearly linear relative speedups, for example, 15.62 on 16 nodes, and essentially linear scaleup in the size of the data set and in the number of clusters desired. For a 2 gigabyte test data set, our implementation drives the 16 node SP2 at more than 1.8 gigaflops.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Agrawal, R., Shafer, J.C.: Parallel mining of association rules: Design, implementation, and experience. IEEE Trans. Knowledge and Data Eng. 8 (1996) 962–969

    Article  Google Scholar 

  2. Chattratichat, J., Darlington, J., Ghanem, M., Guo, Y., Hüning, H., Köhler, M., Sutiwaraphun, J., To, H.W., Yang, D.: Large scale data mining: Challenges and responses. In Pregibon, D., Uthurusamy, R., eds.: Proceedings Third International Conference on Knowledge Discovery and Data Mining, Newport Beach, CA, AAAI Press (1997) 61–64

    Google Scholar 

  3. Cheung, D.W., Xiao, Y.: Effect of data distribution in parallel mining of associations. Data Mining and Knowledge Discovery (1999) to appear.

    Google Scholar 

  4. Han, E.H., Karypis, G., Kumar, V.: Scalable parallel data mining for association rules. In: SIGMOD Record: Proceedings of the 1997 ACM-SIGMOD Conference on Management of Data, Tucson, AZ, USA. (1997) 277–288

    Google Scholar 

  5. Joshi, M.V., Karypis, G., Kumar, V.: ScalParC: A new scalable and efficient parallel classification algorithm for mining large datasets. In: Proceedings of the First Merged International Parallel Processing Symposium and Symposium on Parallel and Distributed Processing, Orlando, FL, USA. (1998) 573–579

    Google Scholar 

  6. Kargupta, H., Hamzaoglu, I., Stafford, B., Hanagandi, V., Buescher, K.: PADMA: Parallel data mining agents for scalable text classification. In: Proceedings of the High Performance Computing, Atlanta, GA, USA. (1997) 290–295

    Google Scholar 

  7. Shafer, J., Agrawal, R., Mehta, M.: A scalable parallel classifier for data mining. In: Proc. 22nd International Conference on VLDB, Mumbai, India. (1996)

    Google Scholar 

  8. Srivastava, A., Han, E.H., Kumar, V., Singh, V.: Parallel formulations of decision-tree classification algorithms. In: Proc. 1998 International Conference on Parallel Processing. (1998)

    Google Scholar 

  9. Zaki, M.J., Ho, C.T., Agrawal, R.: Parallel classification for data mining on shared-memory multiprocessors. In: 15th IEEE Intl. Conf. on Data Engineering. (1999)

    Google Scholar 

  10. Zaki, M.J., Parthasarathy, S., Ogihara, M., Li, W.: New parallel algorithms for fast discovery of association rule. Data Mining and Knowledge Discovery 1 (1997) 343–373

    Article  Google Scholar 

  11. Stolorz, P., Musick, R.: Scalable High Performance Computing for Knowledge Discovery and Data Mining. Kluwer Academic Publishers (1997)

    Google Scholar 

  12. Freitas, A.A., Lavington, S.H.: Mining Very Large Databases with Parallel Processing. Kluwer Academic Publishers (1998)

    Google Scholar 

  13. Hartigan, J.A.: Clustering Algorithms. Wiley (1975)

    Google Scholar 

  14. Fayyad, U.M., Piatetsky-Shapiro, G., Smyth, P., Uthurusamy, R.: Advances in Knowledge Discovery and Data Mining. AAAI/MIT Press (1996)

    Google Scholar 

  15. Fukunaga, K., Narendra, P.M.: A branch and bound algorithm for computing k-nearest neighbors. IEEE Trans. Comput. (1975) 750–753

    Google Scholar 

  16. Cheeseman, P., Stutz, J.: Bayesian classification (autoclass): Theory and results. In Fayyad, U.M., Piatetsky-Shapiro, G., Smyth, P., Uthurusamy, R., eds.: Advances in Knowledge Discovery and Data Mining, AAAI/MIT Press (1996) 153–180

    Google Scholar 

  17. Smyth, P., Ghil, M., Ide, K., Roden, J., Fraser, A.: Detecting atmospheric regimes using cross-validated clustering. In Pregibon, D., Uthurusamy, R., eds.: Proceedings Third International Conference on Knowledge Discovery and Data Mining, Newport Beach, CA, AAAI Press (1997) 61–64

    Google Scholar 

  18. Gersho, A., Gray, R.M.: Vector quantization and signal compression. Kluwer Academic Publishers (1992)

    Google Scholar 

  19. Shaw, C.T., King, G.P.: Using cluster analysis to classify time series. Physica D 58 (1992) 288–298

    Article  Google Scholar 

  20. Dhillon, I.S., Modha, D.S., Spangler, W.S.: Visualizing class structure of multidimensional data. In Weisberg, S., ed.: Proceedings of the 30th Symposium on the Interface: Computing Science and Statistics, Minneapolis, MN. (1998)

    Google Scholar 

  21. Dhillon, I.S., Modha, D.S., Spangler, W.S.: Visualizing class structure of high-dimensional data with applications. Submitted for publication (1999)

    Google Scholar 

  22. Broder, A.Z., Glassman, S.C., Manasse, M.S., Zweig, G.: Syntactic clustering of the web. Technical Report 1997-015, Digital Systems Research Center (1997)

    Google Scholar 

  23. Rasmussen, E.: Clustering algorithms. In Frakes, W.B., Baeza-Yates, R., eds.: Information Retrieval: Data Structures and Algorithms, Prentice Hall, Englewood Cliffs, New Jersey (1992) 419–442

    Google Scholar 

  24. Willet, P.: Recent trends in hierarchic document clustering: a critical review. Inform. Proc. & Management (1988) 577–597

    Google Scholar 

  25. Boley, D., Gini, M., Gross, R., Han, E.H., Hastings, K., Karypis, G., Kumar, V., Mobasher, B., Moore, J.: Document categorization and query generation on the World Wide Web using WebACE. AI Review (1998)

    Google Scholar 

  26. Cutting, D.R., Karger, D.R., Pedersen, J.O., Tukey, J.W.: Scatter/gather: A cluster-based approach to browsing large document collections. In: ACM SIGIR. (1992)

    Google Scholar 

  27. Sahami, M., Yusufali, S., Baldonado, M.: SONIA: A service for organizing net-worked information autonomously. In: ACM Digital Libraries. (1999)

    Google Scholar 

  28. Silverstein, C., Pedersen, J.O.: Almost-constant-time clustering of arbitrary corpus subsets. In: ACM SIGIR. (1997)

    Google Scholar 

  29. Zamir, O., Etzioni, O.: Web document clustering: A feasibility demonstration. In: ACM SIGIR. (1998)

    Google Scholar 

  30. Dhillon, I.S., Modha, D.S.: Concept decompositions for large sparse text data using clustering. Technical Report RJ 10147 (95022), IBM Almaden Research Center (July 8, 1999)

    Google Scholar 

  31. Duda, R.O., Hart, P.E.: Pattern Classification and Scene Analysis. Wiley (1973)

    Google Scholar 

  32. Gropp, W., Lusk, E., Skjellum, A.: Using MPI: Portable Parallel Programming with the Message Passing Interface. The MIT Press, Cambridge, MA (1996)

    Google Scholar 

  33. Snir, M., Otto, S.W., Huss-Lederman, S., Walker, D.W., Dongarra, J.: MPI: The Complete Reference. The MIT Press, Cambridge, MA (1997)

    Google Scholar 

  34. Garey, M.R., Johnson, D.S., Witsenhausen, H.S.: The complexity of the generalized Lloyd-Max problem. IEEE Trans. Inform. Theory 28 (1982) 255–256

    Article  MATH  MathSciNet  Google Scholar 

  35. SAS Institute Cary, NC, USA: SAS Manual. (1997)

    Google Scholar 

  36. Zhang, T., Ramakrishnan, R., Livny, M.: Birch: An efficient data clustering method for very large databases. In: Proceedings of the ACM SIGMOD Conference on Management of Data, Montreal, Canada. (1996)

    Google Scholar 

  37. Bottou, L., Bengio, Y.: Convergence properties of the k-means algorithms. In Tesauro, G., Touretzky, D., eds.: Advances in Neural Information Processing Systems 7, The MIT Press, Cambridge, MA (1995) 585–592

    Google Scholar 

  38. Culler, D.E., Karp, R.M., Patterson, D., Sahay, A., Santos, E.E., Schauser, K.E., Subramonian, R., von Eicken, T.: LogP: A practical model of parallel computation. Communications of the ACM 39 (1996) 78–85

    Article  Google Scholar 

  39. Snir, M., Hochschild, P., Frye, D.D., Gildea, K.J.: The communication software and parallel environment of the IBM SP2. IBM Systems Journal 34 (1995) 205–221

    Article  Google Scholar 

  40. Milligan, G.: An algorithm for creating artificial test clusters. Psychometrika 50 (1985) 123–127

    Article  Google Scholar 

  41. Dubin, D.: clusgen.c. http://alexia.lis.uiuc.edu/~dubin/ (1996)

  42. Northrup, C.J.: Programming with UNIX Threads. John Wiley & Sons (1996)

    Google Scholar 

  43. McLachlan, G.J., Krishnan, T.: The EM Algorithm and Extentions. Wiley (1996)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Texas, Austin, TX, 78712, USA

    Inderjit S. Dhillon

  2. IBM Almaden Research Center, 650 Harry Road, San Jose, CA, 95120, USA

    Dharmendra S. Modha

Authors
  1. Inderjit S. Dhillon
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dharmendra S. Modha
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Computer Science Department, Rensselaer Polytechnic Institute, Troy, NY, 12180, USA

    Mohammed J. Zaki

  2. K55/B1, IBM Almaden Research Center, 650 Harry Road, San Jose, CA, 95120, USA

    Ching-Tien Ho

Rights and permissions

Reprints and permissions

Copyright information

© 2002 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Dhillon, I.S., Modha, D.S. (2002). A Data-Clustering Algorithm on Distributed Memory Multiprocessors. In: Zaki, M.J., Ho, CT. (eds) Large-Scale Parallel Data Mining. Lecture Notes in Computer Science(), vol 1759. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-46502-2_13

Download citation

  • DOI: https://doi.org/10.1007/3-540-46502-2_13

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-67194-7

  • Online ISBN: 978-3-540-46502-7

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation