COMPASSION: A parallel I/O runtime system including chunking and compression for irregular applications
In this paper we present two designs, namely, “Collective I/O” and “Pipelined Collective I/O”, of a runtime library for irregular applications based on the two-phase collective 1/O technique. We also present the optimization of both models by using chunking and compression mechanisms. In the first scheme, all processors participate in compressions and I/O at the same time, making scheduling of I/O requests simpler but creating a possibility of contention at the I/O nodes. In the second approach, processors are grouped into several groups, overlapping communication, compression, and I/O to reduce I/O contention dynamically. Finally, evaluation results are shown that demonstrates that we can obtain significantly high-performance for I/O above what has been possible so far.
Unable to display preview. Download preview PDF.
- 1.Alok Choudhary, Rajesh Bordawekar, Michael Harry, Rakesh Krishnaiyer, Ravi Ponnusamy, Tarvinder Singh, and Rajeev Thakur. PASSION: parallel and scalable software for input-output. Technical Report SCCS-636, ECE Dept., NPAC and CASE Center, Syracuse University, September 1994.Google Scholar
- 2.T. Mattson and G. Henry. The asci option red supercomputer. In Intel Supercomputer Users Group. Thirteenth Annual Conference, Albuquerque, USA, June 1997.Google Scholar
- 3.J. No and A. Choudhary. Techniques to provide run-time support for solving irregular problems. In Proceedings of the International Conference on Parallel and Distributed Systems, Seoul, Korea, 1997.Google Scholar
- 4.R. Ponnusamy, J. Saltz, A. Choudhary, Y.-S. Hwang, and G. Fox. Runtime-compilation techniques for data partitioning and communication schedule reuse. In Proc. of Supercomputing'93, Portland, OR., November 1993.Google Scholar
- 5.K. E. Seamons and M. Winslett. A data management approach for handling large compressed arrays in high performance computing. In Proceedings of the Fifth Symposium on the Frontiers of Massively Parallel Computation, pages 119–128, February 1995.Google Scholar