The Journal of Supercomputing

, Volume 36, Issue 3, pp 219–234

Hierarchical parallel processing of large scale data clustering on a PC cluster with GPU co-processing

Article

DOI: 10.1007/s11227-006-8294-1

Cite this article as:
Takizawa, H. & Kobayashi, H. J Supercomput (2006) 36: 219. doi:10.1007/s11227-006-8294-1

Abstract

This paper presents an effective scheme for clustering a huge data set using a PC cluster system, in which each PC is equipped with a commodity programmable graphics processing unit (GPU). The proposed scheme is devised to achieve three-level hierarchical parallel processing of massive data clustering. The divide-and-conquer approach to parallel data clustering is employed to perform the coarse-grain parallel processing by multiple PCs with a message passing mechanism. By taking advantage of the GPU’s parallel processing capability, moreover, the proposed scheme can exploit two types of the fine-grain data parallelism at the different levels in the nearest neighbor search, which is the most computationally-intensive part of the data-clustering process. The performance of our scheme is discussed in comparison with that of the implementation entirely running on CPU. Experimental results clearly show that the proposed hierarchial parallel processing can remarkably accelerate the data clustering task. Especially, GPU co-processing is quite effective to improve the computational efficiency of parallel data clustering on a PC cluster. Although data-transfer from GPU to CPU is generally costly, acceleration by GPU co-processing is significant to save the total execution time of data-clustering.

Keywords

Programmable graphics processing unit (GPU) General-purpose computation on GPU (GPGPU) k-means data clustering PC cluster The divide-and-conquer approach 

Copyright information

© Springer Science + Business Media, LLC 2006

Authors and Affiliations

  1. 1.Graduate School of Infortmation SciencesTohoku UniversitySendaiJapan
  2. 2.Infortmation Synergy CenterTohoku UniversitySendaiJapan

Personalised recommendations