Dense Affinity Propagation on Clusters of GPUs

  • Marcin Kurdziel
  • Krzysztof Boryczko
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7203)


This article focuses on implementation of Affinity Propagation, a state of the art method for finding exemplars in sets of patterns, on clusters of Graphical Processing Units. When finding exemplars in dense, non-metric data Affinity Propagation has O(n 2) memory complexity. This limits the size of problems that can fit in the Graphical Processing Unit memory. We show, however, that dense Affinity Propagation can be distributed on multiple Graphical Processing Units with low communication-to-computation ratio. By exploiting this favorable communication pattern we propose an implementation which can find exemplars in large, dense data sets efficiently, even when run over slow interconnect.


Affinity Propagation multi-GPU implementation clustering 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Anderson, D., Luke, R., Keller, J.: Incorporation of non-euclidean distance metrics into fuzzy clustering on graphics processing units. In: Melin, P., Castillo, O., Ramrez, E., Kacprzyk, J., Pedrycz, W. (eds.) Analysis and Design of Intelligent Systems using Soft Computing Techniques. AISC, vol. 41, pp. 128–139. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  2. 2.
    Cao, F., Tung, A.K.H., Zhou, A.: Scalable Clustering Using Graphics Processors. In: Yu, J.X., Kitsuregawa, M., Leong, H.-V. (eds.) WAIM 2006. LNCS, vol. 4016, pp. 372–384. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  3. 3.
    Catanzaro, B.: OpenCL optimization case study: Simple reductions. White paper, AMD Developer Central (2010),
  4. 4.
    Charikar, M., Guha, S., Tardos, É., Shmoys, D.: A constant–factor approximation algorithm for the k–median problem. Journal of Computer and System Sciences 65(1), 129–149 (2002)MathSciNetzbMATHCrossRefGoogle Scholar
  5. 5.
    Frey, B., Dueck, D.: Clustering by passing messages between data points. Science 315(5814), 972–976 (2007)MathSciNetzbMATHCrossRefGoogle Scholar
  6. 6.
    Hall, J., Hart, J.: GPU acceleration of iterative clustering. In: The ACM Workshop on General Purpose Computing on Graphics Processors. Manuscript Accompanying Poster at GP2 (2004)Google Scholar
  7. 7.
    Hussein, M., Abd-Almageed, W.: Efficient band approximation of gram matrices for large scale kernel methods on GPUs. In: Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis, SC 2009. ACM, New York (2009)Google Scholar
  8. 8.
    Ma, W., Agrawal, G.: A translation system for enabling data mining applications on GPUs. In: Proceedings of the 23rd International Conference on Supercomputing, pp. 400–409. ACM (2009)Google Scholar
  9. 9.
    MacQueen, J.: Some methods for classification and analysis of multivariate observations. In: Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, pp. 281–297. University of California Press, Berkeley (1967)Google Scholar
  10. 10.
    Message Passing Interface Forum: MPI: A message passing interface standard (1995)Google Scholar
  11. 11.
    Pevsner, J.: Bioinformatics and functional genomics. Wiley-Blackwell (2009)Google Scholar
  12. 12.
    Shalom, S.A.A., Dash, M., Tue, M.: Efficient K-Means Clustering Using Accelerated Graphics Processors. In: Song, I.-Y., Eder, J., Nguyen, T.M. (eds.) DaWaK 2008. LNCS, vol. 5182, pp. 166–175. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  13. 13.
    Takizawa, H., Kobayashi, H.: Hierarchical parallel processing of large scale data clustering on a PC cluster with GPU co-processing. The Journal of Supercomputing 36, 219–234 (2006)CrossRefGoogle Scholar
  14. 14.
    Wu, R., Zhang, B., Hsu, M.: Clustering billions of data points using GPUs. In: Proceedings of the Combined Workshops on UnConventional High Performance Computing Workshop Plus Memory Access Workshop, pp. 1–6. ACM (2009)Google Scholar
  15. 15.
    Zhang, Q., Zhang, Y.: Hierarchical clustering of gene expression profiles with graphics hardware acceleration. Pattern Recognition Letters 27(6), 676–681 (2006)CrossRefGoogle Scholar
  16. 16.
    Zhang, Y., Mueller, F., Cui, X., Potok, T.: Large-scale multi-dimensional document clustering on GPU clusters. In: 2010 IEEE International Symposium on Parallel & Distributed Processing (IPDPS), pp. 1–10. IEEE (2010)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Marcin Kurdziel
    • 1
  • Krzysztof Boryczko
    • 1
  1. 1.Faculty of Electrical Engineering, Automatics, Computer Science and Electronics, Department of Computer ScienceAGH University of Science and TechnologyKrakowPoland

Personalised recommendations