Abstract
With the ever-increasing numbers of cores per node on HPC systems, applications are increasingly using threads to exploit the shared memory within a node, combined with MPI across nodes. Achieving high performance when a large number of concurrent threads make MPI calls is a challenging task for an MPI implementation. We describe the design and implementation of our solution in MPICH2 to achieve high-performance multithreaded communication on the IBM Blue Gene/P. We use a combination of a multichannel-enabled network interface, fine-grained locks, lock-free atomic operations, and specially designed queues to provide a high degree of concurrent access while still maintaining MPI’s message-ordering semantics. We present performance results that demonstrate that our new design improves the multithreaded message rate by a factor of 3.6 compared with the existing implementation on the BG/P. Our solutions are also applicable to other high-end systems that have parallel network access capabilities.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Bailey, D., Harris, T., Saphir, W., Wijngaart, R.V.D., Woo, A., Yarrow, M.: The NAS parallel benchmarks 2.0. NAS Technical Report NAS-95-020, NASA Ames Research Center, Moffett Field, CA (1995)
Balaji, P., Buntinas, D., Goodell, D., Gropp, W., Thakur, R.: Fine-grained multithreading support for hybrid threaded MPI programming. International Journal of High Performance Computing Applications 24(1), 49–57 (2010)
Gropp, W., Thakur, R.: Thread safety in an MPI implementation: Requirements and analysis. Parallel Computing 33(9), 595–604 (2007)
IBM System Blue Gene solution: Blue Gene/P application development , http://www.redbooks.ibm.com/redbooks/pdfs/sg247287.pdf
Kumar, S., Dozsa, G., Almasi, G., Heidelberger, P., Chen, D., Giampapa, M.E., Blocksome, M., Faraj, A., Parker, J., Ratterman, J., Smith, B., Archer, C.J.: The Deep Computing Messaging Framework: Generalized scalable message passing on the Blue Gene/P supercomputer. In: Proceedings of the 22nd International Conference on Supercomputing, pp. 94–103. ACM Press, New York (2008)
Message Passing Interface Forum: MPI: A Message-Passing Interface Standard, Version 2.2 (September 2009), http://www.mpi-forum.org
Sequoia benchmark codes, https://asc.llnl.gov/sequoia/benchmarks/
Snir, M.: MPI-3 hybrid programming proposal, version 7, http://meetings.mpi-forum.org/mpi3.0_hybrid.php
Thakur, R., Gropp, W.: Test suite for evaluating performance of multithreaded MPI communication. Parallel Computing 35(12), 608–617 (2009)
Wijngaart, R.V.D., Jin, H.: NAS parallel benchmarks, multi-zone versions. NAS Technical Report NAS-03-010, NASA Ames Research Center, Moffett Field, CA (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Dózsa, G. et al. (2010). Enabling Concurrent Multithreaded MPI Communication on Multicore Petascale Systems. In: Keller, R., Gabriel, E., Resch, M., Dongarra, J. (eds) Recent Advances in the Message Passing Interface. EuroMPI 2010. Lecture Notes in Computer Science, vol 6305. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-15646-5_2
Download citation
DOI: https://doi.org/10.1007/978-3-642-15646-5_2
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-15645-8
Online ISBN: 978-3-642-15646-5
eBook Packages: Computer ScienceComputer Science (R0)