Abstract
In collective I/O, MPI processes exchange requests so that the rearranged requests can result in the shortest file system access time. Scheduling the exchange sequence determines the response time of participating processes. Existing implementations that simply follow the increasing order of file offsets do not necessary produce the best performance. To minimize the average response time, we propose three scheduling algorithms that consider the number of processes per file stripe and the number of accesses per process. Our experimental results demonstrate improvements of up to 50% in the average response time using two synthetic benchmarks and a high-resolution climate application.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
del Rosario, J., Brodawekar, R., Choudhary, A.: Improved Parallel I/O via a Two-Phase Run-time Access Strategy. In: The Workshop on I/O in Parallel Computer Systems at IPPS (1993)
Kotz, D.: Disk-directed I/O for MIMD Multiprocessors. ACM Transactions on Computer Systems 15(1), 41–74 (1997)
Seamons, K., Chen, Y., Jones, P., Jozwiak, J., Winslett, M.: Server-directed Collective I/O in Panda. In: Supercomputing (November 1995)
Thakur, R., Gropp, W., Lusk, E.: Users Guide for ROMIO. Technical Report ANL/MCS-TM-234, Argonne National Laboratory (October 1997)
Thakur, R., Gropp, W., Lusk, E.: Data Sieving and Collective I/O in ROMIO. In: The Symposium on the Frontiers of Massively Parallel Computation (1999)
Ying, L.: Lustre ADIO Collective Write Driver. Lustre Technical White Paper (September 2008)
Liao, W., Choudhary, A.: Dynamically Adapting File Domain Partitioning Methods for Collective I/O Based on Underlying Parallel File System Locking Protocols. In: SuperComputing Conference (2008)
Liao, W.: Design and Evaluation of MPI File Domain Partitioning Methods under Extent-Based File Locking Protocol. IEEE Transactions on Parallel and Distributed Systems 22(2), 260–272 (2011)
Randall, D., Khairoutdinov, M., Arakawa, A., Grabowski, W.: Breaking the Cloud Parameterization Deadlock. Bull. Amer. Meteor. Soc. 84, 1547–1564 (2003)
Schuchardt, K., Palmer, B., Daily, J., Elsethagen, T., Koontz, A.: IO Strategies and Data Services for Petascale Data Sets from a Global Cloud Resolving Model. Journal of Physics: Conference Series 78 (2007)
Li, J., et al.: Parallel netCDF: A High-Performance Scientific I/O Interface. In: SuperComputing Conference (2003)
Jain, R., Somalwar, K., Werth, J., Browne, J.: Scheduling Parallel I/O Operations in Multiple Bus Systems. Journal of Parallel and Distributed Computing 16(4), 352–362 (1992)
Durand, D., Jain, A., Tseytlin, D.: Applying Randomized Edge Coloring Algorithms to Distributed Communication: An Experimental Study. In: SPAA (1995)
Wu, J., Lin, Y., Liu, P.: Efficient Distributed Algorithms for Parallel I/O Scheduling. In: International Conference on Parallel and Distributed Systems (2005)
Isaila, F., Singh, D., Carretero, J., Garcia, F.: On Evaluating Decentralized Parallel I/O Scheduling Strategies for Parallel File Systems. In: VECPAR (2006)
Chaarawi, M., Chandok, S., Gabriel, E.: Performance Evaluation of Collective Write Algorithms in MPI I/O. In: Allen, G., Nabrzyski, J., Seidel, E., van Albada, G.D., Dongarra, J., Sloot, P.M.A. (eds.) ICCS 2009. LNCS, vol. 5544, pp. 185–194. Springer, Heidelberg (2009)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Jin, C., Sehrish, S., Liao, Wk., Choudhary, A., Schuchardt, K. (2011). Improving the Average Response Time in Collective I/O. In: Cotronis, Y., Danalis, A., Nikolopoulos, D.S., Dongarra, J. (eds) Recent Advances in the Message Passing Interface. EuroMPI 2011. Lecture Notes in Computer Science, vol 6960. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-24449-0_10
Download citation
DOI: https://doi.org/10.1007/978-3-642-24449-0_10
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-24448-3
Online ISBN: 978-3-642-24449-0
eBook Packages: Computer ScienceComputer Science (R0)