Abstract
An MPI-IO library, ROMIO, improves I/O performance for noncontiguous accesses by using its two-phase I/O optimization. When we have multiple MPI processes on each node which has multicore processors, a data aggregation assignment mismatch leads to performance degradation due to network contention. In this paper, we propose an alternative aggregation scheme that manages the striping layout of a Lustre file system to minimize network contention. The optimization has achieved up to about 30 % performance improvements on our 4-node PC cluster system connected via InfiniBand FDR links in performance evaluation by an HPIO benchmark program.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Lustre ADIO collective write driver. Technical report, Lustre (September 2008)
Cha, K., Maeng, S.: An efficient I/O aggregator assignment scheme for collective I/O considering processor affinity. In: Sheu, J., Wang, C. (eds.) 2011 International Conference on Parallel Processing Workshops, ICPPW 2011, September 13–16 2011, pp. 380–388. IEEE Computer Society, Taipei, Taiwan (2011)
Chen, Y., Sun, X.H., Thakur, R., Roth, P.C., Gropp, W.D.: LACIO: a new collective I/O strategy for parallel I/O systems. In: Proceedings of the 25th IEEE International Parallel and Distributed Processing Symposium (IPDPS 2011), pp. 794–804. IEEE Computer Society (2011)
Ching, A., Choudhary, A., keng Liao, W., Ward, L., Pundit, N.: Evaluating I/O characteristics and methods for storing structured scientific data. In: Proceedings 20th IEEE International Parallel and Distributed Processing Symposium, p. 49. IEEE Computer Society (2006)
Liu, J., Crysler, B., Lu, Y., Chen, Y.: Locality-driven high-level i/o aggregation for processing scientific datasets. In: IEEE International Conference on BigData 2013, pp. 103–111. IEEE (2013)
Parallel netCDF. http://cucis.ece.northwestern.edu/projects/PnetCDF/
Thakur, R., Gropp, W., Lusk, E.: An abstract-device interface for implementing portable parallel-I/O interfaces. In: Proceedings of the Sixth Symposium on the Frontiers of Massively Parallel Computation, pp. 180–187 (1996)
Thakur, R., Gropp, W., Lusk, E.: On implementing MPI-IO portably and with high performance. In: Proceedings of the Sixth Workshop on Input/Output in Parallel and Distributed Systems, pp. 23–32 (1999)
The National Center for Supercomputing Applications. http://hdf.ncsa.uiuc.edu/HDF5/
The Ohio State University: MVAPICH: MPI over InfiniBand, 10GigE/iWARP and RoCE. http://mvapich.cse.ohio-state.edu/index.shtml
Acknowledgment
Part of this research work was supported by JSPS KAKENHI Grant Number 25330148. The authors would like to thank members of the System Software Research Team at RIKEN AICS for their useful comments.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Tsujita, Y., Hori, A., Ishikawa, Y. (2015). Striping Layout Aware Data Aggregation for High Performance I/O on a Lustre File System. In: Kunkel, J., Ludwig, T. (eds) High Performance Computing. ISC High Performance 2015. Lecture Notes in Computer Science(), vol 9137. Springer, Cham. https://doi.org/10.1007/978-3-319-20119-1_21
Download citation
DOI: https://doi.org/10.1007/978-3-319-20119-1_21
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-20118-4
Online ISBN: 978-3-319-20119-1
eBook Packages: Computer ScienceComputer Science (R0)