Skip to main content
SpringerLink
Log in

Concurrent programming constructs for parallel MPI applications

The MPI threads library

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Concurrency and parallelism have long been viewed as important, but somewhat distinct concepts. While concurrency is extensively used to amortize latency (for example, in web- and database-servers, user interfaces, etc.), parallelism is traditionally used to enhance performance through execution on multiple functional units. Motivated by an evolving application mix and trends in hardware architecture, there has been a push toward integrating traditional programming models for concurrency and parallelism. Use of conventional threads APIs (POSIX, OpenMP) with messaging libraries (MPI), however, leads to significant programmability concerns, owing primarily to their disparate programming models. In this paper, we describe a novel API and associated runtime for concurrent programming, called MPI Threads (MPIT), which provides a portable and reliable abstraction of low-level threading facilities. We describe various design decisions in MPIT, their underlying motivation, and associated semantics. We provide performance measurements for our prototype implementation to quantify overheads associated with various operations. Finally, we discuss two real-world use cases: an asynchronous message queue and a parallel information retrieval system. We demonstrate that MPIT provides a versatile, low overhead programming model that can be leveraged to program large parallel ensembles.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Adomavicius G, Tuzhilin A (2005) Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions. IEEE Trans Knowl Data Eng 17:734–749

    Article  Google Scholar 

  2. Alam SR, Kuehn JA, Barrett RF, Larkin JM, Fahey MR, Sankaran R, Worley PH (2007) Cray XT4: an early evaluation for petascale scientific simulation. In: Proc SC’07. ACM, New York, pp 39:1–39:12

    Google Scholar 

  3. Balaji P, Buntinas D, Goodell D, Gropp W, Thakur R (2010) Fine-grained multithreading support for hybrid threadd MPI programming. Int J High Perform Comput Appl 24:49–57

    Article  Google Scholar 

  4. Ben-Ari M (2006) Principles of concurrent and distributed programming, 2nd edn. Addison-Wesley, Reading

    Google Scholar 

  5. Berka T, Hagenauer H, Vajteršic M (2011) Portable explicit threading and concurrent programming for MPI applications. In: Proc PPAM. Springer, Berlin, pp 36–47

    Google Scholar 

  6. Berka T, Vajteršic M (2011) Parallel retrieval of dense vectors in the vector space model. CAI 30(2)

  7. Dagum L, Menon R (1998) OpenMP: an industry standard API for shared-memory programming. IEEE Comput Sci Eng 5(1):46–55

    Article  Google Scholar 

  8. Deerwester SC, Dumais ST, Landauer TK, Furnas GW, Harshman RA (1990) Indexing by latent semantic analysis. JSIS 41(6):391–407

    Google Scholar 

  9. Duran A, Gonzàlez M, Corbalán J (2005) Automatic thread distribution for nested parallelism in OpenMP. In: Proc ICS’05. ACM, New York, pp 121–130

    Google Scholar 

  10. Goudy S (2005) A preliminary analysis of the MPI queue characteristics of several applications. In: Proc ICPP. IEEE Press, New York, pp 175–183. doi:10.1109/ICPP.2005.13

    Google Scholar 

  11. Graham RL, Bosilca G (2009) MPI forum: preview of the MPI 3 standard

  12. Hoefler T, Kambadur P, Graham RL, Shipman G, Lumsdaine A (2007) A case for standard non-blocking collective operations. In: Proc Euro. PVM/MPI, France

    Google Scholar 

  13. Hybrid Programming Working Group (2011) MPI3: the behavior of MPI processes sharing memory, available at http://lists.mpi-forum.org/mpi3-hybridpm/2011/10/0414.php (last retrieved November 22, 2011)

  14. Jolliffe IT (2002) Principal component analysis, 2nd edn. Springer, Berlin

    MATH  Google Scholar 

  15. Kobayashi M, Aono M, Takeuchi H, Samukawa H (2002) Matrix computations for information retrieval and major and outlier cluster detection. JCAM 149(1):119–129

    MathSciNet  MATH  Google Scholar 

  16. Lusk E, Chan A (2008) Early experiments with the OpenMP/MPI hybrid programming model. In: Proc IWOMP. Springer, Berlin, pp 36–47

    Google Scholar 

  17. Message Passing Interface Forum (2009) MPI: a message-passing interface standard, version 2.2

  18. OpenMP Architecture Review Board (2008) The OpenMP API specification for parallel programming

  19. Orlando S, Perego R, Silvestri F (2004) Design of a parallel and distributed web search engine. ArXiv Computer Science e-prints

  20. Saad Y (2011) Numerical methods for large eigenvalue problems, 2nd edn. SIAM, Philadelphia

    Book  MATH  Google Scholar 

  21. Smith L, Bull M (2001) Development of mixed mode MPI/OpenMP applications. SPJ 9:83–98

    Google Scholar 

  22. Thakur R, Gropp W (2009) Test suite for evaluating performance of multithreaded MPI communication. Parallel Comput 35:608–617

    Article  Google Scholar 

  23. The Austin Group, The Open Group, The IEEE (2009) ISO/IEC/IEEE 9945:2009 information technology—portable operating system interface (POSIX©) base specifications, issue 7

Download references

Author information

Authors and Affiliations

  1. Department of Computer Sciences, University of Salzburg, Jakob-Haringer-Str. 2, 5020, Salzburg, Austria

    Tobias Berka, Helge Hagenauer & Marian Vajteršic

  2. Department of Computer Science, Purdue University, 305 N. University Street, West Lafayette, IN, 47907, USA

    Tobias Berka, Giorgos Kollias & Ananth Grama

  3. Department of Informatics, Mathematical Institute, Slovak Academy of Sciences, Stefanikova 49, 814 73, Bratislava, Slovakia

    Marian Vajteršic

Authors
  1. Tobias Berka
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Giorgos Kollias
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Helge Hagenauer
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Marian Vajteršic
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ananth Grama
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tobias Berka.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Berka, T., Kollias, G., Hagenauer, H. et al. Concurrent programming constructs for parallel MPI applications. J Supercomput 63, 385–406 (2013). https://doi.org/10.1007/s11227-011-0739-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-011-0739-5

Keywords

Search

Navigation