Chapter

High Performance Computing - HiPC 2008

Volume 5374 of the series Lecture Notes in Computer Science pp 323-335

ScELA: Scalable and Extensible Launching Architecture for Clusters

  • Jaidev K. SridharAffiliated withNetwork-Based Computing Laboratory, The Ohio State University
  • , Matthew J. KoopAffiliated withNetwork-Based Computing Laboratory, The Ohio State University
  • , Jonathan L. PerkinsAffiliated withNetwork-Based Computing Laboratory, The Ohio State University
  • , Dhabaleswar K. PandaAffiliated withNetwork-Based Computing Laboratory, The Ohio State University

* Final gross prices may vary according to local VAT.

Get Access

Abstract

As cluster sizes head into tens of thousands, current job launch mechanisms do not scale as they are limited by resource constraints as well as performance bottlenecks. The job launch process includes two phases – spawning of processes on processors and information exchange between processes for job initialization. Implementations of various programming models follow distinct protocols for the information exchange phase. We present the design of a scalable, extensible and high-performance job launch architecture for very large scale parallel computing. We present implementations of this architecture which achieve a speedup of more than 700% in launching a simple Hello World MPI application on 10,240 processor cores and also scale to more than 3 times the number of processor cores compared to prior solutions.