Pattern Matching and I/O Replay for POSIX I/O in Parallel Programs

  • Michael Kluge
  • Andreas Knüpfer
  • Matthias Müller
  • Wolfgang E. Nagel
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5704)


This paper describes an approach to track, compress and replay I/O calls from a parallel application. We will introduce our method to capture calls to POSIX I/O functions and to create I/O dependency graphs for parallel program runs. The resulting graphs will be compressed by a pattern matching algorithm and used as an input for a flexible I/O benchmark. The first parts of the paper cover the information gathering and compression in the area of POSIX I/O while the later parts are dedicated to the I/O benchmark and to some results from experiments.


Pattern Match Parallel Program Compression Algorithm Spectral Element Compression Scheme 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Alam, S.R., Vetter, J.S.: A framework to develop symbolic performance models of parallel applications. In: Parallel and Distributed Processing Symposium, IPDPS 2006 (April 2006)Google Scholar
  2. 2.
    Knüpfer, A., Brendel, R., Brunst, H., Mix, H., Nagel, W.E.: Introducing the Open Trace Format (OTF). In: Alexandrov, V.N., van Albada, G.D., Sloot, P.M.A., Dongarra, J. (eds.) ICCS 2006. LNCS, vol. 3992, pp. 526–533. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  3. 3.
    Bailey, D., Barszcz, E., Barton, J., Browning, D., Carter, R., Dagum, L., Fatoohi, R., Fineberg, S., Frederickson, P., Lasinski, T., Schreiber, R., Simon, H., Venkatakrishnan, V., Weeratunga, S.: NAS Parallel Benchmarks (2007),
  4. 4.
    Bergroth, L., Hakonen, H., Raita, T.: A survey of longest common subsequence algorithms. In: Proceedings of Seventh International Symposium on String Processing and Information Retrieval, 2000. SPIRE 2000, pp. 39–48 (2000)Google Scholar
  5. 5.
    Blackburn, H.M., Sherwin, S.J.: Formulation of a Galerkin spectral element-Fourier method for three-dimensional incompressible flows in cylindrical geometries. J. Comp. Phys. 197, 759–778 (2004)CrossRefzbMATHGoogle Scholar
  6. 6.
    Ching, A., Choudhary, A., Liao, W., Ward, L., Pundit, N.: Evaluating I/O Characteristics and Methods for Storing Structured Scientific Data. In: Proceedings of IPDPS (2006)Google Scholar
  7. 7.
    Chinner, D., Higdon, J.: Exploring High Bandwidth Filesystems on Large Systems. Technical report, Ottawa Linux Symposium, SGI (2006)Google Scholar
  8. 8.
    Cocker, R.: Bonnie++ web page (2001),
  9. 9.
    Parallel I/O Benchmarking Consortium. Parallel I/O Benchmarking Consortium web site (2007),
  10. 10.
    Darling, A.E., Carey, L., Feng, W.c.: The design, implementation, and evaluation of mpiblast. In: 4th International Conference on Linux Clusters: The HPC Revolution 2003 (2003)Google Scholar
  11. 11.
    Gómez-Villamor, S., Muntés-Mulero, V., Pérez-Casany, M., Tran, J., Rees, S., Larriba-Pey, J.-L.: IOAgent: A parallel I/O workload generator. In: Nagel, W.E., Walter, W.V., Lehner, W. (eds.) Euro-Par 2006. LNCS, vol. 4128, pp. 3–14. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  12. 12.
    Koal, K., Stiller, J., Grundmann, R.: Linear and nonlinear instability in a cylindrical enclosure caused by a rotating magnetic field. Phys. Fluids 19, 088107 (2007)CrossRefzbMATHGoogle Scholar
  13. 13.
    Krietemeyer, M., Merz, M.: IPACS-Benchmark - Integrated Performance Analysis of Computer Systems (IPACS). Logos Verlag, Berlin (2006)Google Scholar
  14. 14.
    Loewe, W., Morrone, C.: IOR benchmark (2007),
  15. 15.
    May, J.: Pinola: A script-based i/o benchmark. In: Petascale Data Storage Workshop at SC 2008 (2008)Google Scholar
  16. 16.
    Mesnier, M.P., Wachs, M., Sambasivan, R.R., Lopez, J., Hendricks, J., Ganger, G.R., O’Hallaron, D.R.: Trace: parallel trace replay with approximate causal events. In: FAST 2007: Proceedings of the 5th USENIX conference on File and Storage Technologies, p. 24. USENIX Association (2007)Google Scholar
  17. 17.
    Noeth, M., Mueller, F., Schulz, M., de Supinski, B.R.: Scalable compression and replay of communication traces in massively parallel environments. In: IPDPS, pp. 1–11 (2007)Google Scholar
  18. 18.
    Norcott, W.D., Capps, D.: IOzone benchmark (2006),
  19. 19.
    Rabenseifner, R., Koniges, A.E.: Effective File-I/O Bandwidth Benchmark. In: Bode, A., Ludwig, T., Karl, W.C., Wismüller, R. (eds.) Euro-Par 2000. LNCS, vol. 1900, p. 1273. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  20. 20.
    ZIH TU Dresden. VampirTrace (2007),
  21. 21.
    Veillard, D.: The XML C parser and toolkit (2007),
  22. 22.
    Wrigth, C.P., Aranya, A., Zadok, E.: Tracefs: A file system to trace them all. In: Proceedings of the Third USENIX Conference on File and Storage Technologies (FAST 2004), pp. 129–143. USENIX Association (2004)Google Scholar
  23. 23.
    Zingale, M.: FLASH I/O benchmark routine – parallel HDF 5 (2001),

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Michael Kluge
    • 1
  • Andreas Knüpfer
    • 1
  • Matthias Müller
    • 1
  • Wolfgang E. Nagel
    • 1
  1. 1.Technische Universität DresdenDresdenGermany

Personalised recommendations