International Journal of Parallel Programming

, Volume 44, Issue 3, pp 466–485

Parallel Tree Accumulations on MapReduce

Article

DOI: 10.1007/s10766-015-0355-8

Cite this article as:
Matsuzaki, K. & Miyazaki, R. Int J Parallel Prog (2016) 44: 466. doi:10.1007/s10766-015-0355-8

Abstract

MapReduce is a remarkable parallel programming model as well as a parallel processing infrastructure for large-scale data processing. Since it is now widely available on cloud environments, developing methodology or patterns for MapReduce programming is important. In particular, XML is the de facto standard for representing data, and processing semi-structured data is involved in many applications. The target computational patterns in this paper are tree accumulations. Tree accumulations are shape-preserving computations over a tree in which values are updated through flows over the tree. We develop BSP algorithms for two tree accumulations as extensions of the BSP algorithm for tree reduction by Kakehi et al. (Tech. Rep. METR 2006-64, Department of Mathematical Informatics, Graduate School of Information Science and Technology, The University of Tokyo, 2006). We also implemented the two-superstep algorithms with a single MapReduce execution. Experimental results on a 16-node PC cluster show good speedups of a factor of 10.9–12.7.

Keywords

Tree accumulations MapReduce Hadoop Bulk synchronous parallel (BSP) model 

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  1. 1.School of InformationKochi University of TechnologyKamiJapan