Skip to main content

Efficient Parallel Multi-way Merging on Heterogeneous Multi-core Cluster

  • Conference paper
  • 3285 Accesses

Part of the Communications in Computer and Information Science book series (CCIS,volume 405)

Abstract

Data merging and sorting are often applied to the scientific and engineering applications such as computational fluid dynamics and computation geometry. By constructing a data sending matrix, solving the data exchange range and determining the data exchange order among compute nodes to reduce the communication overhead, this paper proposes a load-balance data distribution strategy among nodes, and designs a communication-efficient parallel multi-way merging algorithm on the heterogeneous cluster with the multi-core compute nodes which have different computation speed, communication rate and memory capacity. The experimental results on the heterogeneous cluster with multi-core machines show that the proposed parallel merging algorithm obtains high speedup and has good scalability.

Keywords

  • Parallel merging
  • Parallel sorting
  • Multi-core architectures
  • Heterogeneous cluster
  • Thread-level parallelism

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-642-53962-6_10
  • Chapter length: 12 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   84.99
Price excludes VAT (USA)
  • ISBN: 978-3-642-53962-6
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   109.99
Price excludes VAT (USA)

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Chen, G.L.: Design and analysis for parallel algorithms. Higher Education Press, Beijing (2009)

    Google Scholar 

  2. Helman, D.R., Joseph, J.: Sorting on clusters of SMPs. In: Proceedings of the 12th International Parallel Processing Symposium, pp. 561–582. IEEE Computer Society Press, Los Alamitos (1998)

    Google Scholar 

  3. Ding, W.Q., Ji, Y.C., Chen, G.L.: A Parallel Merging Algorithm Based on MPP. Journal of Computer Research and Development 36(1), 52–56 (1999)

    Google Scholar 

  4. Giusti, A.D., Naiouf, M., Chichizola, F., et al.: Dynamic Load Balance in Parallel Merge Sorting over Homogeneous Clusters. In: Proceedings of the 19th International Conference on Advanced Information Networking and Applications, vol. 2, pp. 219–222. IEEE Computer Society Press, Los Alamitos (2005)

    Google Scholar 

  5. Jeon, M., Kim, D.: Parallel Merge Sort with Load Balancing. International Journal of Parallel Programming 31(1), 411–434 (2003)

    CrossRef  Google Scholar 

  6. Govindaraju, N., Gray, J., Kumar, R.: GPUTerasort: High Performance Graphics Coprocessor Sorting for Large Database Management. In: Proceedings of the 2006 ACM SIGMOD International Conference on Management of Data, pp. 325–336. ACM Press, New York (2006)

    CrossRef  Google Scholar 

  7. Gedik, B., Bordawekar, R.R., Yu, P.S.: CellSort: High Performance Sorting on the Cell Processor. In: Proceedings of 33rd International Conference on Very Large Data Bases, Vienna, Austria, September 23-27, pp. 1286–1297 (2007)

    Google Scholar 

  8. Chhugani, J., Nguyen, A.D., Lee, V.W., et al.: Efficient implementation of Sorting on Multi-core SIMD CPU architecture. In: Proceedings of 34th International Conference on Very Large Data Bases, Auckland, New Zealand, August 23-28, pp. 1313–1324 (2008)

    Google Scholar 

  9. Hao, S., Du, Z., Bader, D., et al.: A Partition-Merge Based Cache-Conscious Parallel Sorting Algorithm for CMP with Shared Cache. In: Proceedings of International Conference on Parallel Processing, Vienna, Austria, September 22-25, pp. 396–403 (2009)

    Google Scholar 

  10. Hultén, R., Kessler, C.W., Keller, J.: Optimized On-Chip-Pipelined Mergesort on the Cell/B.E. In: D’Ambra, P., Guarracino, M., Talia, D. (eds.) Euro-Par 2010, Part II. LNCS, vol. 6272, pp. 187–198. Springer, Heidelberg (2010)

    CrossRef  Google Scholar 

  11. Satish, N., Kim, C., Chhugani, J., et al.: Fast sort on CPUs and GPUs: a case for bandwidth oblivious SIMD sort. In: Proceedings of the 2010 International Conference on Management of Data, Indianapolis, Indiana, USA, June 6-11, pp. 351–362 (2010)

    Google Scholar 

  12. Zhong, C., Qu, Z.Y., Yang, F., Yin, M.X., Li, X.: Efficient and Scalable Thread-level Parallel Algorithms for Sorting Multisets on Multi-core Systems. Journal of Computers 7(1), 30–41 (2012)

    CrossRef  Google Scholar 

  13. Zhong, C., Li, X., Yang, F., Liu, J., Yin, M., Huang, Y.: Scheduling Divisible Loads with Return Messages on Multi-core Heterogeneous Clusters with Unknown System Parameters. International Journal of Advancements in Computing Technology 4(7), 110–120 (2012)

    CrossRef  Google Scholar 

  14. Wei, W.: Study on Parallel Merging Algorithms on Multi-core Systems. Master Degree Thesis, Guangxi University (2011)

    Google Scholar 

  15. Francis, R., Mathieson, I., Pannan, L.: A Fast, Simple Algorithm to Balance a Parallel Multiway Merge. In: Proceedings of 5th International PARLE Conference, Munich, Germany, June 14-17, pp. 570–581 (1993)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2014 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Zhong, C., Wei, W. (2014). Efficient Parallel Multi-way Merging on Heterogeneous Multi-core Cluster. In: Li, K., Xiao, Z., Wang, Y., Du, J., Li, K. (eds) Parallel Computational Fluid Dynamics. ParCFD 2013. Communications in Computer and Information Science, vol 405. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-53962-6_10

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-53962-6_10

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-53961-9

  • Online ISBN: 978-3-642-53962-6

  • eBook Packages: Computer ScienceComputer Science (R0)