Constructing MPI Input-output Datatypes for Efficient Transpacking

  • Faisal Ghias Mir
  • Jesper Larsson Träff
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5205)

Abstract

Communication and file I/O buffers in MPI can contain contiguous as well as non-contiguous, structured data. To describe non-consecutive data layouts compactly, MPI provides a powerful concept of derived or user-defined datatypes. Especially for MPI-IO, where data are transferred between file and memory buffers, the need for copying between differently typed MPI buffers arise. A straightforward solution to this typed copy problem consists in packing and unpacking the differently structured data via an intermediate buffer. For a maximally efficient MPI(-IO) implementation, means of copying directly (without intermediate buffers) between differently typed MPI buffers are needed.

We present a new approach to the typed copy problem. For any two given MPI datatypes describing the layout of input and output buffer, respectively, we show how to efficiently construct an input-output type that subsumes both. This type is used to copy directly from input to output buffer by means of a special transpack function. By completely eliminating the need for intermediate buffering, the typed copy performance can in theory be improved by up to a factor of two, with only a modest overhead for constructing the input-output type. An experimental evaluation shows that even more significant improvements can be achieved in practice.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Faisal Ghias Mir
    • 1
  • Jesper Larsson Träff
    • 1
  1. 1.NEC Laboratories Europe, NEC Europe Ltd.Sankt AugustinGermany

Personalised recommendations