Evaluating the Performance and Scalability of MapReduce Applications on X10

  • Chao Zhang
  • Chenning Xie
  • Zhiwei Xiao
  • Haibo Chen
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6965)

Abstract

MapReduce has been shown to be a simple and efficient way to harness the massive resources of clusters. Recently, researchers propose using partitioned global address space (PGAS) based language and runtime to ease the programming of large-scale clusters. In this paper, we present an empirical study on the effectiveness of running MapReduce applications on a typical PGAS language runtime called X10. By tuning the performance of two applications on X10 platforms, we successfully eliminate several performance bottlenecks related to I/O processing. We also identify several remaining problems and propose several approaches to remedying them. Our final performance evaluation on a small-scale multicore cluster shows that the MapReduce applications written with X10 notably outperform those in Hadoop in most cases. Detailed analysis reveals that the major performance advantages come from a simplified task management and data storage scheme.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Chao Zhang
    • 1
  • Chenning Xie
    • 1
  • Zhiwei Xiao
    • 1
  • Haibo Chen
    • 1
  1. 1.Parallel Processing InstituteFudan UniversityChina

Personalised recommendations