COMPASSION: A parallel I/O runtime system including chunking and compression for irregular applications

  • Jesús Carretero
  • Jaechun No
  • Sung-soon Park
  • Alok Choudhary
  • Pang Chen
3. Computer Science
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1401)


In this paper we present two designs, namely, “Collective I/O” and “Pipelined Collective I/O”, of a runtime library for irregular applications based on the two-phase collective 1/O technique. We also present the optimization of both models by using chunking and compression mechanisms. In the first scheme, all processors participate in compressions and I/O at the same time, making scheduling of I/O requests simpler but creating a possibility of contention at the I/O nodes. In the second approach, processors are grouped into several groups, overlapping communication, compression, and I/O to reduce I/O contention dynamically. Finally, evaluation results are shown that demonstrates that we can obtain significantly high-performance for I/O above what has been possible so far.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Alok Choudhary, Rajesh Bordawekar, Michael Harry, Rakesh Krishnaiyer, Ravi Ponnusamy, Tarvinder Singh, and Rajeev Thakur. PASSION: parallel and scalable software for input-output. Technical Report SCCS-636, ECE Dept., NPAC and CASE Center, Syracuse University, September 1994.Google Scholar
  2. 2.
    T. Mattson and G. Henry. The asci option red supercomputer. In Intel Supercomputer Users Group. Thirteenth Annual Conference, Albuquerque, USA, June 1997.Google Scholar
  3. 3.
    J. No and A. Choudhary. Techniques to provide run-time support for solving irregular problems. In Proceedings of the International Conference on Parallel and Distributed Systems, Seoul, Korea, 1997.Google Scholar
  4. 4.
    R. Ponnusamy, J. Saltz, A. Choudhary, Y.-S. Hwang, and G. Fox. Runtime-compilation techniques for data partitioning and communication schedule reuse. In Proc. of Supercomputing'93, Portland, OR., November 1993.Google Scholar
  5. 5.
    K. E. Seamons and M. Winslett. A data management approach for handling large compressed arrays in high performance computing. In Proceedings of the Fifth Symposium on the Frontiers of Massively Parallel Computation, pages 119–128, February 1995.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1998

Authors and Affiliations

  • Jesús Carretero
    • 1
  • Jaechun No
    • 2
  • Sung-soon Park
    • 3
  • Alok Choudhary
    • 4
  • Pang Chen
    • 5
  1. 1.Arquitectura y Tecnología de Sistemas InformáticosUniversidad Politécnica de MadridSpain
  2. 2.Dept. of Electrical Engineering and Computer ScienceSyracuse UniversityUSA
  3. 3.Electrical and Computer EngineeringNorthwestern UniversityUSA
  4. 4.Computer Science Dept.Anyang UniversityUSA
  5. 5.Sandia National LaboratoriesUSA

Personalised recommendations