Advertisement

Performance Comparison of Graph BFS Implemented in MapReduce and PGAS Programming Models

  • Magdalena Ryczkowska
  • Marek Nowicki
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10778)

Abstract

Computations based on graphs are very common problems but complexity, increasing size of analyzed graphs and a huge amount of communication make this analysis a challenging task. In this paper, we present a comparison of two parallel BFS (Breath-First Search) implementations: MapReduce run on Hadoop infrastructure and in PGAS (Partitioned Global Address Space) model. The latter implementation has been developed with the help of the PCJ (Parallel Computations in Java) - a library for parallel and distributed computations in Java. Both implementations realize the level synchronous strategy - Hadoop algorithm assumes iterative MapReduce jobs, whereas PCJ uses explicit synchronization after each level. The scalability of both solutions is similar. However, the PCJ implementation is much faster (about 100 times) than the MapReduce Hadoop solution.

Keywords

High performance computing Hadoop MapReduce PGAS Parallel and distributed computation Performance evaluation Parallel graph algorithms Java 

Notes

Acknowledgments

This work has been performed using the PL-Grid infrastructure. The authors would like to thank CHIST-ERA consortium for financial support under HPDCJ project (Polish part funded by NCN grant 2014/14/Z/ST6/00007).

References

  1. 1.
    Dean, J., Ghemawat, S.: MapReduce: simplified data processing on large clusters. Commun. ACM 51(1), 107–113 (2008)CrossRefGoogle Scholar
  2. 2.
    Apache Hadoop. http://hadoop.apache.org/. Accessed 20 May 2017
  3. 3.
    Nowicki, M., Bała, P.: Parallel computations in Java with PCJ library. In: Smari, W.W., Zeljkovic, V. (eds.) 2012 International Conference on High Performance Computing and Simulation (HPCS), pp. 381–387. IEEE (2012)Google Scholar
  4. 4.
    Nowicki, M., Bała, P.: PCJ - new approach for parallel computations in Java. In: Manninen, P., Öster, P. (eds.) PARA 2012. LNCS, vol. 7782, pp. 115–125. Springer, Heidelberg (2013).  https://doi.org/10.1007/978-3-642-36803-5_8 CrossRefGoogle Scholar
  5. 5.
    http://pcj.icm.edu.pl. Accessed 25 May 2017
  6. 6.
  7. 7.
    Ryczkowska, M., Nowicki, M., Bala, P.: The performance evaluation of the Java implementation of Graph500. In: Wyrzykowski, R., Deelman, E., Dongarra, J., Karczewski, K., Kitowski, J., Wiatr, K. (eds.) PPAM 2015. LNCS, vol. 9574, pp. 221–230. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-32152-3_21 CrossRefGoogle Scholar
  8. 8.
    Ryczkowska, M., Nowicki, M., Bała, P.: Level-synchronous BFS algorithm implemented in Java using PCJ library. In: Proceedings of the 2016 International Conference on Computational Science and Computational Intelligence (CSCI), Las Vegas, NV, USA, pp. 596–601 (2016)Google Scholar
  9. 9.
    Murphy, R.C., Wheeler, K.B., Barrett, B.W., Ang, J.A.: Introducing the Graph500. Cray User’s Group (CUG) (2010)Google Scholar
  10. 10.
    White, T.: Hadoop: The Definitive Guide, 4th edn. O’Reilly, Sebastopol (2015)Google Scholar
  11. 11.
    Altschul, S.F., Gish, W., Miller, W., Myers, E.W., Lipman, D.J.: Basic local alignment search tool. J. Mol. Biol. 215(3), 403–410 (1990)CrossRefGoogle Scholar
  12. 12.
    Suzumura, T., Ueno, K., Sato, H., Fujisawa, K., Matsuoka, S.: Performance characteristics of Graph500 on large-scale distributed environment. In: Proceedings of the 2011 IEEE International Symposium on Workload Characterization (IISWC), pp. 149–158 (2011)Google Scholar
  13. 13.
    Ueno, K., Suzumura, T.: Highly scalable graph search for the Graph500 benchmark, In: Proceedings of the 21st International ACM Symposium on High-Performance Parallel and Distributed Computing, pp. 149–160 (2012)Google Scholar
  14. 14.
    Buluc, A., Madduri, K.: Parallel breadth-first search on distributed memory systems. In: Proceedings of the 2011 International Conference for High Performance Computing, Networking, Storage and Analysis (2011)Google Scholar
  15. 15.
    Berrendorf, R., Makulla, M.: Level-synchronous parallel breadth-first search algorithms for multicore and multiprocessor systems. In: FC 2014, pp. 26–31 (2014)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Interdisciplinary Centre for Mathematical and Computational ModelingUniversity of WarsawWarsawPoland
  2. 2.Faculty of Mathematics and Computer ScienceNicolaus Copernicus UniversityTorunPoland

Personalised recommendations