International Workshop on Advanced Parallel Processing Technologies

APPT 2011: Advanced Parallel Processing Technologies pp 46-57

Evaluating the Performance and Scalability of MapReduce Applications on X10

  • Chao Zhang
  • Chenning Xie
  • Zhiwei Xiao
  • Haibo Chen
Conference paper

DOI: 10.1007/978-3-642-24151-2_4

Volume 6965 of the book series Lecture Notes in Computer Science (LNCS)
Cite this paper as:
Zhang C., Xie C., Xiao Z., Chen H. (2011) Evaluating the Performance and Scalability of MapReduce Applications on X10. In: Temam O., Yew PC., Zang B. (eds) Advanced Parallel Processing Technologies. APPT 2011. Lecture Notes in Computer Science, vol 6965. Springer, Berlin, Heidelberg

Abstract

MapReduce has been shown to be a simple and efficient way to harness the massive resources of clusters. Recently, researchers propose using partitioned global address space (PGAS) based language and runtime to ease the programming of large-scale clusters. In this paper, we present an empirical study on the effectiveness of running MapReduce applications on a typical PGAS language runtime called X10. By tuning the performance of two applications on X10 platforms, we successfully eliminate several performance bottlenecks related to I/O processing. We also identify several remaining problems and propose several approaches to remedying them. Our final performance evaluation on a small-scale multicore cluster shows that the MapReduce applications written with X10 notably outperform those in Hadoop in most cases. Detailed analysis reveals that the major performance advantages come from a simplified task management and data storage scheme.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Chao Zhang
    • 1
  • Chenning Xie
    • 1
  • Zhiwei Xiao
    • 1
  • Haibo Chen
    • 1
  1. 1.Parallel Processing InstituteFudan UniversityChina