Constructing MPI Input-output Datatypes for Efficient Transpacking
- Faisal Ghias MirAffiliated withNEC Laboratories Europe, NEC Europe Ltd.
- , Jesper Larsson TräffAffiliated withNEC Laboratories Europe, NEC Europe Ltd.
Communication and file I/O buffers in MPI can contain contiguous as well as non-contiguous, structured data. To describe non-consecutive data layouts compactly, MPI provides a powerful concept of derived or user-defined datatypes. Especially for MPI-IO, where data are transferred between file and memory buffers, the need for copying between differently typed MPI buffers arise. A straightforward solution to this typed copy problem consists in packing and unpacking the differently structured data via an intermediate buffer. For a maximally efficient MPI(-IO) implementation, means of copying directly (without intermediate buffers) between differently typed MPI buffers are needed.
We present a new approach to the typed copy problem. For any two given MPI datatypes describing the layout of input and output buffer, respectively, we show how to efficiently construct an input-output type that subsumes both. This type is used to copy directly from input to output buffer by means of a special transpack function. By completely eliminating the need for intermediate buffering, the typed copy performance can in theory be improved by up to a factor of two, with only a modest overhead for constructing the input-output type. An experimental evaluation shows that even more significant improvements can be achieved in practice.
- Constructing MPI Input-output Datatypes for Efficient Transpacking
- Book Title
- Recent Advances in Parallel Virtual Machine and Message Passing Interface
- Book Subtitle
- 15th European PVM/MPI Users’ Group Meeting, Dublin, Ireland, September 7-10, 2008. Proceedings
- pp 141-150
- Print ISBN
- Online ISBN
- Series Title
- Lecture Notes in Computer Science
- Series Volume
- Series ISSN
- Springer Berlin Heidelberg
- Copyright Holder
- Springer-Verlag Berlin Heidelberg
- Additional Links
- Industry Sectors
- eBook Packages
To view the rest of this content please follow the download PDF link above.