The Impact of File Systems on MPI-IO Scalability

  • Rob Latham
  • Rob Ross
  • Rajeev Thakur
Conference paper

DOI: 10.1007/978-3-540-30218-6_18

Volume 3241 of the book series Lecture Notes in Computer Science (LNCS)
Cite this paper as:
Latham R., Ross R., Thakur R. (2004) The Impact of File Systems on MPI-IO Scalability. In: Kranzlmüller D., Kacsuk P., Dongarra J. (eds) Recent Advances in Parallel Virtual Machine and Message Passing Interface. EuroPVM/MPI 2004. Lecture Notes in Computer Science, vol 3241. Springer, Berlin, Heidelberg

Abstract

As the number of nodes in cluster systems continues to grow, leveraging scalable algorithms in all aspects of such systems becomes key to maintaining performance. While scalable algorithms have been applied successfully in some areas of parallel I/O, many operations are still performed in an uncoordinated manner. In this work we consider, in three file system scenarios, the possibilities for applying scalable algorithms to the many operations that make up the MPI-IO interface. From this evaluation we extract a set of file system characteristics that aid in developing scalable MPI-IO implementations.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Rob Latham
    • 1
  • Rob Ross
    • 1
  • Rajeev Thakur
    • 1
  1. 1.Argonne National LaboratoryArgonneUSA