Processing MPI Datatypes Outside MPI

  • Robert Ross
  • Robert Latham
  • William Gropp
  • Ewing Lusk
  • Rajeev Thakur
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5759)

Abstract

The MPI datatype functionality provides a powerful tool for describing structured memory and file regions in parallel applications, enabling noncontiguous data to be operated on by MPI communication and I/O routines. However, no facilities are provided by the MPI standard to allow users to efficiently manipulate MPI datatypes in their own codes.

We present MPITypes, an open source, portable library that enables the construction of efficient MPI datatype processing routines outside the MPI implementation. MPITypes enables programmers who are not MPI implementors to create efficient datatype processing routines. We show the use of MPITypes in three examples: copying data between user buffers and a “pack” buffer, encoding of data in a portable format, and transpacking. Our experimental evaluation shows that the implementation achieves rates comparable to existing MPI implementations.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Thakur, R., Gropp, W., Lusk, E.: On implementing MPI-IO portably and with high performance. In: Proceedings of the 6th Workshop on I/O in Parallel and Distributed Systems, pp. 23–32. ACM Press, New York (1999)Google Scholar
  2. 2.
    Li, J., Liao, W.-k., Choudhary, A., Ross, R., Thakur, R., Gropp, W., Latham, R., Siegel, A., Gallagher, B., Zingale, M.: Parallel netCDF: A high-performance scientific I/O interface. In: Proceedings of SC2003: High Performance Networking and Computing, Phoenix, AZ. IEEE Computer Society Press, Los Alamitos (2003)Google Scholar
  3. 3.
    Ross, R.J., Miller, N., Gropp, W.D.: Implementing fast and reusable datatype processing. In: Dongarra, J., Laforenza, D., Orlando, S. (eds.) EuroPVM/MPI 2003. LNCS, vol. 2840, pp. 404–413. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  4. 4.
    Mir, F., Träff, J.: Constructing MPI input-output datatypes for efficient transpacking. In: Lastovetsky, A., Kechadi, T., Dongarra, J. (eds.) EuroPVM/MPI 2008. LNCS, vol. 5205, pp. 141–150. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  5. 5.
    Rew, R., Davis, G.: The unidata netCDF: Software for scientific data access. In: Sixth International Conference on Interactive Information and Processing Systems for Meteorology, Oceanography and Hydrology, February 1990, pp. 33–40 (1990)Google Scholar
  6. 6.
    Message Passing Interface Forum: MPI-2: Extensions to the message-passing interface (July 1997), http://www.mpi-forum.org/docs/docs.html
  7. 7.
    Worringen, J., Träff, J.L., Ritzdorf, H.: Improving generic non-contiguous file access for MPI-IO. In: Dongarra, J., Laforenza, D., Orlando, S. (eds.) EuroPVM/MPI 2003. LNCS, vol. 2840, pp. 309–318. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  8. 8.
    Träff, J., Hempel, R., Ritzdoff, H., Zimmermann, F.: Flattening on the fly: Efficient handling of MPI derived datatypes. In: Margalef, T., Dongarra, J., Luque, E. (eds.) PVM/MPI 1999. LNCS, vol. 1697, pp. 109–116. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  9. 9.
    Gropp, W., Lusk, E., Skjellum, A.: Using MPI: Portable Parallel Programming with the Message-Passing Interface. MIT Press, Cambridge (1994)MATHGoogle Scholar
  10. 10.
    Fryxell, B., Olson, K., Ricker, P., Timmes, F.X., Zingale, M., Lamb, D.Q., MacNeice, P., Rosner, R., Tufo, H.: FLASH: An adaptive mesh hydrodynamics code for modelling astrophysical thermonuclear flashes. Astrophysical Journal Supplement 131, 273–334 (2000)CrossRefGoogle Scholar
  11. 11.
    Thakur, R., Gropp, W., Lusk, E.: Data sieving and collective I/O in ROMIO. In: Proceedings of the 7th Symposium on the Frontiers of Massively Parallel Computation, pp. 182–189. IEEE Computer Society Press, Los Alamitos (1999)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Robert Ross
    • 1
  • Robert Latham
    • 1
  • William Gropp
    • 2
  • Ewing Lusk
    • 1
  • Rajeev Thakur
    • 1
  1. 1.Mathematics and Computer Science DivisionArgonne National LaboratoryArgonneUSA
  2. 2.Computer Science DepartmentUniversity of Illinois at Urbana-ChampaignUrbanaUSA

Personalised recommendations