Performance Expectations and Guidelines for MPI Derived Datatypes

  • William Gropp
  • Torsten Hoefler
  • Rajeev Thakur
  • Jesper Larsson Träff
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6960)


MPI’s derived datatypes provide a powerful mechanism for concisely describing arbitrary, noncontiguous layouts of user data for use in MPI communication. This paper formulates self-consistent performance guidelines for derived datatypes. Such guidelines make performance expectations for derived datatypes explicit and suggest relevant optimizations to MPI implementers. We also identify self-consistent guidelines that are too strict to enforce, because they entail NP-hard optimization problems. Enforced self-consistent guidelines assure the user that certain manual datatype optimizations cannot lead to performance improvements, which in turn contributes to performance portability between MPI implementations that behave in accordance with the guidelines. We present results of tests with several MPI implementations, which indicate that many of them violate the guidelines.


Performance Expectation Performance Portability Type Constructor Communication Operation Performance Guideline 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bajrović, E., Träff, J.L.: Using MPI derived datatypes in numerical libraries. In: Cotronis, Y., et al. (eds.) EuroMPI 2011. LNCS, vol. 6960, pp. 29–38. Springer, Heidelberg (2011)Google Scholar
  2. 2.
    Byna, S., Sun, X.-H., Thakur, R., Gropp, W.D.: Automatic memory optimizations for improving MPI derived datatype performance. In: Mohr, B., Träff, J.L., Worringen, J., Dongarra, J. (eds.) PVM/MPI 2006. LNCS, vol. 4192, pp. 238–246. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  3. 3.
    Gropp, W.D., Kimpe, D., Ross, R., Thakur, R., Träff, J.L.: Self-consistent MPI-IO performance requirements and expectations. In: Lastovetsky, A., Kechadi, T., Dongarra, J. (eds.) EuroPVM/MPI 2008. LNCS, vol. 5205, pp. 167–176. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  4. 4.
    Gropp, W.D., Lusk, E., Swider, D.: Improving the performance of MPI derived datatypes. In: Proceedings of the Third MPI Developer’s and User’s Conference, pp. 25–30. MPI Software Technology Press (1999)Google Scholar
  5. 5.
    Hoefler, T., Gottlieb, S.: Parallel zero-copy algorithms for fast fourier transform and conjugate gradient using MPI datatypes. In: Keller, R., Gabriel, E., Resch, M., Dongarra, J. (eds.) EuroMPI 2010. LNCS, vol. 6305, pp. 132–141. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  6. 6.
    Lu, Q., Wu, J., Panda, D.K., Sadayappan, P.: Applying MPI derived datatypes to the NAS benchmarks: A case study. In: 33rd International Conference on Parallel Processing Workshops (ICPP 2004 Workshops), pp. 538–545. IEEE Computer Society, Los Alamitos (2004)Google Scholar
  7. 7.
    MPI Forum: MPI: A Message-Passing Interface Standard. Version 2.2 ( September 4, 2009),
  8. 8.
    Reussner, R., Träff, J.L., Hunzelmann, G.: A benchmark for MPI derived datatypes. In: Dongarra, J., Kacsuk, P., Podhorszki, N. (eds.) PVM/MPI 2000. LNCS, vol. 1908, pp. 10–17. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  9. 9.
    Ross, R., Miller, N., Gropp, W.D.: Implementing fast and reusable datatype processing. In: Dongarra, J., Laforenza, D., Orlando, S. (eds.) EuroPVM/MPI 2003. LNCS, vol. 2840, pp. 404–413. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  10. 10.
    Santhanaraman, G., Wu, J., Huang, W., Panda, D.K.: Designing zero-copy message passing interface derived datatype communication over Infiniband: Alternative approaches and performance evaluation. International Journal on High Performance Computing Applications 19(2), 129–142 (2005)CrossRefGoogle Scholar
  11. 11.
    Träff, J.L., Gropp, W.D., Thakur, R.: Self-consistent MPI performance guidelines. IEEE Transactions on Parallel and Distributed Systems 21(5), 698–709 (2010)CrossRefGoogle Scholar
  12. 12.
    Träff, J.L., Hempel, R., Ritzdorf, H., Zimmermann, F.: Flattening on the fly: Efficient handling of MPI derived datatypes. In: Margalef, T., Dongarra, J., Luque, E. (eds.) PVM/MPI 1999. LNCS, vol. 1697, pp. 109–116. Springer, Heidelberg (1999)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • William Gropp
    • 1
  • Torsten Hoefler
    • 1
  • Rajeev Thakur
    • 2
  • Jesper Larsson Träff
    • 3
  1. 1.University of IllinoisUrbanaUSA
  2. 2.Argonne National LaboratoryArgonneUSA
  3. 3.University of ViennaAustria

Personalised recommendations