The Impact of File Systems on MPI-IO Scalability

  • Rob Latham
  • Rob Ross
  • Rajeev Thakur
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3241)


As the number of nodes in cluster systems continues to grow, leveraging scalable algorithms in all aspects of such systems becomes key to maintaining performance. While scalable algorithms have been applied successfully in some areas of parallel I/O, many operations are still performed in an uncoordinated manner. In this work we consider, in three file system scenarios, the possibilities for applying scalable algorithms to the many operations that make up the MPI-IO interface. From this evaluation we extract a set of file system characteristics that aid in developing scalable MPI-IO implementations.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    ALC, the ASCI Linux Cluster,
  2. 2.
    Ching, A., Choudhary, A., Coloma, K., Liao, W.k., Ross, R., Gropp, W.: Noncontiguous I/O accesses through MPI-IO. In: Proceedings of the Third IEEE/ACM International Symposium on Cluster Computing and the Grid, Tokyo, Japan, May 2003, pp. 104–111. IEEE Computer Society Press, Los Alamitos (2003)CrossRefGoogle Scholar
  3. 3.
    Ching, A., Choudhary, A., Liao, W.k., Ross, R., Gropp, W.: Noncontiguous I/O through PVFS. In: Proceedings of the 2002 IEEE International Conference on Cluster Computing (September 2002)Google Scholar
  4. 4.
    Ching, A., Choudhary, W., Liao, R.: Efficient structured data access in parallel file systems. In: Proceedings of Cluster 2003, Hong Kong (November 2003)Google Scholar
  5. 5.
    IBM DataStar Cluster,
  6. 6.
    IEEE/ANSI Std. 1003.1. Portable operating system interface (POSIX)–part 1: System application program interface (API) [C language], 1996 edition Google Scholar
  7. 7.
    Isaila, F., Tichy, W.F.: View I/O: Improving the performance of noncontiguous I/O. In: Proceedings of IEEE Cluster Computing Conference, Hong Kong (December 2003)Google Scholar
  8. 8.
    LCRC, the Argonne National Laboratory Computing Project,
  9. 9.
    Ma, X., Winslett, M., Lee, J., Yu, S.: Improving MPI IO output performance with active buffering plus threads. In: Proceedings of the International Parallel and Distributed Processing Symposium, April 2003, IEEE Computer Society Press, Los Alamitos (2003)Google Scholar
  10. 10.
    MPI-2: Extensions to the message-passing interface. The MPI Forum (July 1997)Google Scholar
  11. 11.
    Prost, J.-P., Treumann, R., Hedges, R., Jia, B., Koniges, A.: MPI-IO GPFS, an optimized implementation of MPI-IO on top of GPFS. In: Proceedings of Supercomputing 2001 (November 2001)Google Scholar
  12. 12.
    The Parallel Virtual File System, version 2,
  13. 13.
    Thakur, R., Choudhary, A.: An Extended Two-Phase Method for Accessing Sections of Out-of-Core Arrays. Scientific Programming 5(4), 301–317 (1996)Google Scholar
  14. 14.
    Thakur, R., Gropp, W., Lusk, E.: A case for using MPI’s derived datatypes to improve I/O performance. In: Proceedings of SC98: High Performance Networking and Computing, November 1998, ACM Press, New York (1998)Google Scholar
  15. 15.
    Worringen, J., Traff, J.L., Ritzdorf, H.: Fast parallel noncontiguous file access. In: Proceedings of SC 2003: High Performance Networking and Computing, Phoenix, AZ, November 2003, IEEE Computer Society Press, Los Alamitos (2003)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Rob Latham
    • 1
  • Rob Ross
    • 1
  • Rajeev Thakur
    • 1
  1. 1.Argonne National LaboratoryArgonneUSA

Personalised recommendations