Abstract
In this paper we present the design, implementation and evaluation of a runtime system based on collective I/O techniques for irregular applications. Its main goal is to provide parallel collective I/O, by having all processors participate in the I/O simultaneously, and making the mapping of the I/O requests simpler. Using such a technique, the input/output of the irregular applications can be greatly simplified by always maintaining global files canonically ordered, thus avoiding the utilization of multiple files and the associated sorting/merging steps. The run-time library has been optimized by applying in-memory compression mechanisms to the collective I/O operations. We also present the results of several evaluation experiments obtained by running a particle in cell application on an Intel Paragon machine. Those results demonstrate that significantly high-performance for I/O can be obtained by using our library.
This work was supported in part by NSF Young Investigator Award CCR-9357840 and NSF CCR-9509143.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
G.A. Bird. Molecular Gas Dynamics and the Direct Simulation of Gas Flows. Oxford University Press, 1994.
J. Carretero, J. No, S. Park, A. Choudhary, and P. Chen. Compassion: a parallel i/o runtime system including chunking and compression for irregular applications. In Proceedings of the International Conference on High-Performance Computing and Networking 1998, Amsterdam, Holland, April 1998.
M. Christon, D. Crawford, E. Hertel, J. Peery, and A. Robinson. Asci red-experiences and lessons learned with a massively parallel teraflop supercomputer. In Proceedings of the Supercomputer 1997, Mannheim, Germany, June 1997.
James T. Poole. Preliminary survey of I/O intensive applications. Technical Report CCSF-38, Scalable I/O Initiative, Caltech Concurrent Supercomputing Facilities, Caltech, 1994.
L. A. Schoof and V. R. Yarberry. Exodusii: A finite element data model. Technical Report SAND94-2137, Sandia National Laboratories., December 1994.
K. E. Seamons and M. Winslett. A data management approach for handling large compressed arrays in high performance computing. In Proceedings of the Fifth Symposium on the Frontiers of Massively Parallel Computation, pages 119–128, February 1995.
Shamik D. Sharma, Ravi Ponnusamy, Bongki Moon, Yuan shin Hwang, Raja Das, and Joel Saltz. Run-time and compile-time support for adaptive irregular problems. In Supercomputing 1994. IEEE Press, November 1994.
Rajeev Thakur, Alok Choudhary, Rajesh Bordawekar, Sachin More, and Sivaramakrishna Kuditipudi. Passion: Optimized I/O for parallel applications. IEEE Computer, 29(6):70–78, June 1996.
Rajeev Thakur, Ewing Lusk, and William Gropp. I/O characterization of a portable astrophysics application on the IBM SP and Intel Paragon. Technical Report MCS-P534-0895, Argonne National Laboratory, August 1995. Revised October 1995.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1999 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Carretero, J., No, J., Choudhary, A. (1999). Optimizing I/O for Irregular Applications on Distributed-Memory Machines. In: Zinterhof, P., Vajteršic, M., Uhl, A. (eds) Parallel Computation. ACPC 1999. Lecture Notes in Computer Science, vol 1557. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-49164-3_45
Download citation
DOI: https://doi.org/10.1007/3-540-49164-3_45
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-65641-8
Online ISBN: 978-3-540-49164-4
eBook Packages: Springer Book Archive