Workload characterization of input/output intensive parallel applications

  • Evgenia Smirni
  • Daniel A. Reed
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1245)

Abstract

The broadening disparity in the performance of input/output (I/O) devices and the performance of processors and communication links on parallel systems is a major obstacle to achieving high performance for a wide range of parallel applications. I/O hardware and file system parallelism are the keys to bridging this performance gap. A prerequisite to the development of efficient parallel file systems is detailed characterization of the I/O demands of parallel applications. In this paper, we present a comparative study of the I/O access patterns commonly found in I/O intensive parallel applications. Using the Pablo performance analysis environment and its I/O extensions we captured application I/O access patterns and analyzed their interactions with current parallel I/O systems. This analysis has proven instrumental in guiding the development of new application programming interfaces (APIs) for parallel file systems and in developing effective file system policies that can adaptively respond to complex application I/O requirements.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Bennett, R., Bryant, K., Sussman, A., Das, R., and Saltz, J. Jovian: A framework for optimizing parallel I/O. In Proceedings of the Scalable Parallel Libraries Conference (October 1994), IEEE Computer Society Press, pp. 10–20.Google Scholar
  2. 2.
    Bordawekar, R., Thakur, R., and Choudhary, A. Efficient compilation of out-of-core data parallel programs. Tech. Rep. SCCS-622, NPAC, April 1994.Google Scholar
  3. 3.
    Corbett, P. F., Prost, J.-P., Demetriou, C., Gibson, G., Riedel, E., Zelenka, J., Chen, Y., Felten, E., Li, K., Hartman, J., Peterson, L., Bershad, B., Wolman, A., and Aydt, R. Proposal for a common parallel file system programming interface version 1.0, September 1996.Google Scholar
  4. 4.
    Crandall, P., Aydt, R. A., Chien, A. A., and Reed, D. A. Input/Output characterization of scalable parallel applications. In Supercomputing 1995 (1996).Google Scholar
  5. 5.
    Foster, I., and Nieplocha, J. ChemIO: High-performance I/O for computational chemistry applications, http://www.mcs.anl.gov/chemio/, February 1996.Google Scholar
  6. 6.
    Kotz, D., and Nieuwejaar, N. Dynamic file-access characteristics of a production parallel scientific workload. In Supercomputing '94 (November 1994).Google Scholar
  7. 7.
    Madhyastha, T., and Reed, D. A. Intelligent, adaptive file system policy selection. In Proceedings of Frontiers'96 (1996).Google Scholar
  8. 8.
    Miller, E. L., and Katz, R. H. Input/Output behavior of supercomputer applications. In Supercomputing '91 (November 1991), pp. 567–576.Google Scholar
  9. 9.
    Nieuwejaar, N., and Kotz, D. The Galley parallel file system. In Proceedings of the 10th ACM International Conference on Supercomputing (May 1996).Google Scholar
  10. 10.
    Pasquale, B. K., and Polyzos, G. C. Dynamic I/O characterization of I/O intensive scientific applications. In Proceedings of Supercomputing '94 (November 1994), pp. 660–669.Google Scholar
  11. 11.
    Poole, J. T. Scalable I/O Initiative. California Institute of Technology, Available at http://www.ccsf.caltech.edu/SIO/, 1996.Google Scholar
  12. 12.
    Purakayastha, A., Ellis, C. S., Kotz, D., Nieuwejaar, N., and Best, M. Characterizing parallel file-access patterns on a large-scale multiprocessor. In Proceedings of the Ninth International Parallel Processing Symposium (April 1995), pp. 165–172.Google Scholar
  13. 13.
    Reed, D. A., Aydt, R. A., Noe, R. J., Roth, P. C., Shields, K. A., Schwartz, B. W., and Tavera, L. F. Scalable performance analysis: The Pablo performance analysis environment. In Proceedings of the Scalable Parallel Libraries Conference, A. Skjellum, Ed. IEEE Computer Society, 1993, pp. 104–113.Google Scholar
  14. 14.
    Reed, D. A., Elford, C. L., Madhyastha, T., Scullin, W. H., Aydt, R. A., and Smirni, E. I/O, performance analysis, and performance data immersion. In Proceedings of MASCOTS '96 (Feb. 1996), pp. 1–12.Google Scholar
  15. 15.
    Smirni, E., Aydt, R. A., Chien, A. A., and Reed, D. A. I/O requirements of scientific applications: An evolutionary view. In High Performance Distributed Computing (1996), pp. 49–59.Google Scholar
  16. 16.
    Toledo, S., and Gustavson, F. G. The design and implementation of SOLAR, a portable library for scalable out-of-core linear algebra computations. In Fourth Workshop on Input/Output in Parallel and Distributed Systems (May 1996), pp. 28–40.Google Scholar

Copyright information

© Springer-Verlag 1997

Authors and Affiliations

  • Evgenia Smirni
    • 1
  • Daniel A. Reed
    • 1
  1. 1.Department of Computer ScienceUniversity of IllinoisUrbanaUSA

Personalised recommendations