Skip to main content

Advertisement

SpringerLink
Log in
Menu
Find a journal Publish with us
Search
Cart
Book cover

European Conference on Parallel Processing

Euro-Par 2012: Euro-Par 2012 Parallel Processing pp 77–88Cite as

  1. Home
  2. Euro-Par 2012 Parallel Processing
  3. Conference paper
Understanding I/O Performance Using I/O Skeletal Applications

Understanding I/O Performance Using I/O Skeletal Applications

  • Jeremy Logan19,20,
  • Scott Klasky19,
  • Hasan Abbasi19,
  • Qing Liu19,
  • George Ostrouchov19,
  • Manish Parashar21,
  • Norbert Podhorszki19,
  • Yuan Tian22 &
  • …
  • Matthew Wolf19,23 
  • Conference paper

Part of the Lecture Notes in Computer Science book series (LNTCS,volume 7484)

Abstract

We address the difficulty involved in obtaining meaningful measurements of I/O performance in HPC applications, as well as the further challenge of understanding the causes of I/O bottlenecks in these applications. The need for I/O optimization is critical given the difficulty in scaling I/O to ever increasing numbers of processing cores. To address this need, we have pioneered a new approach to the analysis of I/O performance using automatic generation of I/O benchmark codes given a high-level description of an application’s I/O pattern. By combining this with low-level characterization of the performance of the various components of the underlying I/O method we are able to produce a complete picture of the I/O behavior of an application.

We compare the performance measurements obtained using Skel, the tool that implements our approach, with those of an instrumented version of the original application to show that our approach is accurate. We demonstrate the use of Skel to compare the performance of several I/O methods. Finally we show that the detailed breakdown of timing information produced by Skel provides better understanding of the reasons for the performance differences between the examined I/O methods. We conclude that our approach facilitates faster, more accurate and more meaningful I/O performance testing, allowing application I/O performance to be predicted, and new systems and I/O methods to be evaluated.

Keywords

  • Maximum Throughput
  • Communication Operation
  • Stripe Size
  • Core Count
  • Application Source Code

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Download conference paper PDF

References

  1. Lang, S., Carns, P., Latham, R., Ross, R., Harms, K., Allcock, W.: I/O performance challenges at leadership scale. In: Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis, SC 2009, pp. 40:1–40:12. ACM, New York (2009)

    Google Scholar 

  2. FLASH I/O benchmark routine – parallel HDF5, http://www.ucolick.org/~zingale/flash_benchmark_io/

  3. Chen, J., Choudhary, A., De Supinski, B., DeVries, M., Hawkes, E., Klasky, S., Liao, W., Ma, K., Mellor-Crummey, J., Podhorszki, N., et al.: Terascale direct numerical simulations of turbulent combustion using S3D. Computational Science & Discovery 2, 015001 (2009)

    CrossRef  Google Scholar 

  4. ADIOS 1.3 user’s manual, http://users.nccs.gov/~pnorbert/ADIOS-UsersManual-1.3.pdf

  5. Logan, J., Klasky, S., Lofstead, J., Abbasi, H., Ethier, S., Grout, R., Ku, S.H., Liu, Q., Ma, X., Parashar, M., Podhorszki, N., Schwan, K., Wolf, M.: Skel: generative software for producing skeletal I/O applications. In: The Proceedings of D3science (2011)

    Google Scholar 

  6. Lofstead, J., Zheng, F., Klasky, S., Schwan, K.: Adaptable, metadata rich IO methods for portable high performance IO. In: IEEE International Symposium on Parallel Distributed Processing, IPDPS 2009, pp. 1–10 (May 2009)

    Google Scholar 

  7. Tian, Y., Klasky, S., Abbasi, H., Lofstead, J., Grout, R., Podhorski, N., Liu, Q., Wang, Y., Yu, W.: EDO: improving read performance for scientific applications through elastic data organization. In: Proceedings of IEEE Cluster (2011)

    Google Scholar 

  8. Lofstead, J., Polte, M., Gibson, G., Klasky, S., Schwan, K., Oldfield, R., Wolf, M., Liu, Q.: Six degrees of scientific data: reading patterns for extreme scale science IO. In: Proceedings of the 20th International Symposium on High Performance Distributed Computing, HPDC 2011, pp. 49–60. ACM, New York (2011)

    CrossRef  Google Scholar 

  9. Messer, O.E.B., Bruenn, S.W., Blondin, J.M., Hix, W.R., Mezzacappa, A., Dirk, C.J.: Petascale supernova simulation with CHIMERA. Journal of Physics: Conference Series 78(1), 012049 (2007)

    CrossRef  Google Scholar 

  10. Wang, W.X., Lin, Z., Tang, W.M., Lee, W.W., Ethier, S., Lewandowski, J.L.V., Rewoldt, G., Hahm, T.S., Manickam, J.: Gyro-kinetic simulation of global turbulent transport properties in tokamak experiments. Physics of Plasmas 13(9), 092505 (2006)

    CrossRef  Google Scholar 

  11. Titan configuration and timeline, http://www.olcf.ornl.gov/titan/system-configuration-timeline/

  12. OLCF computing resources: Sith, http://www.olcf.ornl.gov/computing-resources/sith/

  13. Shipman, G., Dillow, D., Oral, S., Wang, F.: The spider center wide file system: From concept to reality. In: Proceedings, Cray User Group (CUG) Conference, Atlanta, GA (2009)

    Google Scholar 

  14. IOR HPC Benchmark, http://sourceforge.net/projects/ior-sio/

  15. Bailey, D., Harris, T., Saphir, W., Van Der Wijngaart, R., Woo, A., Yarrow, M.: The NAS parallel benchmarks 2.0. Technical Report NAS-95-020, NASA Ames Research Center, Tech. rep. (1995)

    Google Scholar 

  16. Bailey, D., Barszcz, E., Barton, J., Browning, D., Carter, R., Dagum, L., Fatoohi, R., Frederickson, P., Lasinski, T., Schreiber, R., et al.: The NAS parallel benchmarks summary and preliminary results. In: Proceedings of the 1991 ACM/IEEE Conference on Supercomputing, pp. 158–165. IEEE (1991)

    Google Scholar 

  17. Shan, H., Shalf, J.: Using IOR to analyze the I/O performance for HPC platforms. In: Cray Users Group Meeting (CUG) 2007, Seattle, Washington (May 2007)

    Google Scholar 

  18. MADbench2, http://crd.lbl.gov/~borrill/MADbench2/

  19. Darshan, petascale I/O characterization tool, http://www.mcs.anl.gov/research/projects/darshan/

  20. Vijayakumar, K., Mueller, F., Ma, X., Roth, P.C.: Scalable I/O tracing and analysis. In: Proceedings of the 4th Annual Workshop on Petascale Data Storage, PDSW 2009, pp. 26–31. ACM, New York (2009)

    CrossRef  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Oak Ridge National Laboratory, Oak Ridge, Tennessee, USA

    Jeremy Logan, Scott Klasky, Hasan Abbasi, Qing Liu, George Ostrouchov, Norbert Podhorszki & Matthew Wolf

  2. University of Tennessee, Knoxville, Tennessee, USA

    Jeremy Logan

  3. Rutgers, The State University of New Jersey, Piscataway, New Jersey, USA

    Manish Parashar

  4. Auburn University, Auburn, Alabama, USA

    Yuan Tian

  5. Georgia Tech., Atlanta, Georgia, USA

    Matthew Wolf

Authors
  1. Jeremy Logan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Scott Klasky
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hasan Abbasi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Qing Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. George Ostrouchov
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Manish Parashar
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Norbert Podhorszki
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Yuan Tian
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Matthew Wolf
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. University of Patras, Computer Technology Institute and Press “Diophantus”,, N. Kazantzaki, 26504, Rio, Greece

    Christos Kaklamanis

  2. University of Patras, University Building B, 26504, Rio, Greece

    Theodore Papatheodorou

  3. Computer Technology Institute and Press “Diophantus”, University of Patras, N. Kazantzaki, 26504, Rio, Greece

    Paul G. Spirakis

Rights and permissions

Reprints and Permissions

Copyright information

© 2012 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Logan, J. et al. (2012). Understanding I/O Performance Using I/O Skeletal Applications. In: Kaklamanis, C., Papatheodorou, T., Spirakis, P.G. (eds) Euro-Par 2012 Parallel Processing. Euro-Par 2012. Lecture Notes in Computer Science, vol 7484. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-32820-6_10

Download citation

  • .RIS
  • .ENW
  • .BIB
  • DOI: https://doi.org/10.1007/978-3-642-32820-6_10

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-32819-0

  • Online ISBN: 978-3-642-32820-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Share this paper

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Search

Navigation

  • Find a journal
  • Publish with us

Discover content

  • Journals A-Z
  • Books A-Z

Publish with us

  • Publish your research
  • Open access publishing

Products and services

  • Our products
  • Librarians
  • Societies
  • Partners and advertisers

Our imprints

  • Springer
  • Nature Portfolio
  • BMC
  • Palgrave Macmillan
  • Apress
  • Your US state privacy rights
  • Accessibility statement
  • Terms and conditions
  • Privacy policy
  • Help and support

167.114.118.210

Not affiliated

Springer Nature

© 2023 Springer Nature