A system level view of Petascale I/O on IBM Blue Gene/P

Open Access
Special Issue Paper


Petascale supercomputers rely on highly efficient Petascale I/O subsystems. This work describes the tuning and scaling behavior of the GPFS parallel file system on JUGENE, the largest IBM Blue Gene/P installation worldwide and the first PetaFlop/s HPC resource within the European PRACE Research Infrastructure.


Parallel I/O Blue Gene GPFS 


  1. 1.
    IBM Blue Gene team (2008) Overview of the IBM Blue Gene/P project. IBM J Res Dev 52(1/2):199–220. Online at http://dx.doi.org/10.1147/rd.521.0199 Google Scholar
  2. 2.
    Gara A, et al. (2005) Overview of the Blue Gene/L system architecture. IBM J Res Dev 49(2/3):213–248. Online at http://dx.doi.org/10.1147/rd.492.0195 Google Scholar
  3. 3.
    Coteus P, et al. (2005) Packaging the Blue Gene/L supercomputer. IBM J Res Dev 49(2/3):213–248. Online at http://dx.doi.org/10.1147/rd.492.0195 CrossRefGoogle Scholar
  4. 4.
    Moreira J, et al. (2005) Blue Gene/L programming and operating environment. IBM J Res Dev 49(2/3):367–376 CrossRefGoogle Scholar
  5. 5.
    Mohr B, Frings W (2010) Jülich Blue Gene/P extreme scaling workshop 2009. Technical report FZJ-JSC-IB-2010-02. Online at http://www.fz-juelich.de/jsc/docs/printable/ib/ib-10/ib-2010-02.pdf
  6. 6.
    Mohr B, Frings W (2010) Jülich Blue Gene/P extreme scaling workshop 2010. Technical report FZJ-JSC-IB-2010-03. Online at http://www.fz-juelich.de/jsc/docs/printable/ib/ib-10/ib-2010-03.pdf
  7. 7.
    Schmuck F, Haskin R (2002) GPFS: A shared-disk file system for large computing clusters. In: Proceedings of the first USENIX conference on file and storage technologies. Monterey, CA, January 28–30, 2002, pp 231–244, 2002. Online at http://www.usenix.org/publications/library/proceedings/fast02/ Google Scholar
  8. 8.
    Mextorf O, Schmidt U, Wollschläger L, Hennecke M, Kutzer K (2010) Storage and network design for the JUGENE Petaflop system. inSiDE 8(1):62–66 Online at http://inside.hlrs.de/htm/editions.htm Google Scholar
  9. 9.
    Vishwanath V, et al. (2010) Accelerating I/O Forwarding in IBM Blue Gene/P Systems. Online at http://www.mcs.anl.gov/uploads/cels/papers/P1745.pdf
  10. 10.
    Latham R (2008) Parallel I/O for high performance and scalability. In: 14th annual meeting of ScicomP, NY, 19–23 May 2008. Online at http://www.spscicomp.org/ScicomP14/talks/Latham.pdf
  11. 11.
    Parallel-NetCDF: a high performance API for NetCDF file access. Online at http://www.mcs.anl.gov/parallel-netcdf/
  12. 12.
    The HDF Group Parallel HDF5. Online at http://www.hdfgroup.org/HDF5/PHDF5/
  13. 13.
    Unidata. Parallel I/O with netCDF-4. Online at http://www.unidata.ucar.edu/software/netcdf/netcdf-4/
  14. 14.
    Gropp W, Huss-Lederman S, Lumsdaine A, Lusk E, Netzberg B, Saphir W, Snir M (1998) MPI: the complete reference. The MPI-2 extensions, vol 2. MIT-Press, Cambridge, p 10 Google Scholar
  15. 15.
    SIONlib: scalable massively parallel I/O to task-local files. Online at http://www.fz-juelich.de/jsc/sionlib/
  16. 16.
    Frings W, Wolf F, Petkov V (2009) Scalable massively parallel I/O to task-local files. In: Proceedings of SC09, Portland, OR, USA, November 14–20. Online at http://dx.doi.org/10.1145/1654059.1654077 Google Scholar
  17. 17.
    GPFS Version 3.3 (2009) Concepts, planning, and installation guide. IBM publication GA76-0413-03, September 2009 Google Scholar
  18. 18.
    GPFS Version 3.3 (2009) Administration and programming reference. IBM publication SC23-2221-03, September 2009 Google Scholar
  19. 19.
    GPFS Version 3.3 Advanced administration guide. IBM publication SC23-5182-03, September 2009 Google Scholar
  20. 20.
    IBM developerWorks “HPC Central” Wiki. Online at http://www.ibm.com/developerworks/wikis/display/hpccentral/
  21. 21.
    Lakner G (2010) IBM system Blue Gene solution: Blue Gene/P system administration. Configuring I/O nodes. Chap 12. IBM redbook SG24-7417-03. Online at http://www.redbooks.ibm.com/abstracts/sg247417.html
  22. 22.
    Hennecke M (2006) GPFS multicluster with the IBM System Blue. Gene solution and eHPS clusters. IBM redpaper REDP4168. Online at http://www.redbooks.ibm.com/abstracts/redp4168.html
  23. 23.
    IOR HPC benchmark. Online at http://sourceforge.net/projects/ior-sio/
  24. 24.
    Shan H, Shalf J (2007) Using IOR to analyze the I/O performance for HPC Platforms. LBNL technical report. Online at http://escholarship.org/uc/item/9111c60j

Copyright information

© The Author(s) 2011

Authors and Affiliations

  1. 1.Forschungszentrum Jülich GmbH (FZJ)JülichGermany
  2. 2.IBM Deutschland GmbHDüsseldorfGermany

Personalised recommendations