Improving Reactivity and Communication Overlap in MPI Using a Generic I/O Manager

  • François Trahay
  • Alexandre Denis
  • Olivier Aumage
  • Raymond Namyst
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4757)


MPI applications may waste thousands of CPU cycles if they do not efficiently overlap communications and computation. In this paper, we present a generic and portable I/O manager that is able to make communication progress asynchronously using tasklets. It chooses automatically the most appropriate communication method, depending on the context: multi-threaded application or not, SMP machine or not. We have implemented and evaluated our I/O manager with Mad-MPI, our own MPI implementation, and compared it to other existing MPI implementations regarding the ability to efficiently overlap communication and computation.


Polling Interrupt Thread Scheduler High-Speed Network 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Aumage, O., Brunet, E., Furmento, N., Namyst, R.: Newmadeleine: a fast communication scheduling engine for high performance networks. In: CAC 2007: Workshop on Communication Architecture for Clusters, held in conjunction with IPDPS 2007 (2007)Google Scholar
  2. 2.
    Sancho, J.C., et al.: Quantifying the potential benefit of overlapping communication and computation in large-scale scientific applications. In: SC 2006, IEEE Computer Society, Los Alamitos (2006)Google Scholar
  3. 3.
    Doerfler, D., Brightwell, R.: Measuring MPI send and receive overhead and application availability in high performance network interfaces. In: Euro PVM/MPI, pp. 331–338 (2006)Google Scholar
  4. 4.
    ANL, MCS Division (2007), MPICH-2 Home Page
  5. 5.
    The Open MPI Project: Open MPI: Open Source High Performance Computing (2007),
  6. 6.
    Graham, R.L., et al.: Open MPI: A high-performance, heterogeneous MPI. In: Proceedings, Fifth International Workshop on Algorithms, Models and Tools for Parallel Computing on Heterogeneous Networks, Barcelona, Spain (2006)Google Scholar
  7. 7.
    Quadrics Ltd.: Elan Programming Manual (2003),
  8. 8.
    Myricom Inc.: Myrinet EXpress (MX): A High Performance, Low-level, Message-Passing Interface for Myrinet (2003),
  9. 9.
    Anderson, T.E., Bershad, B.N., Lazowska, E.D., Levy, H.M.: Scheduler activations: effective kernel support for the user-level management of parallelism. ACM Trans. Comput. Syst. 10(1), 53–79 (1992)CrossRefGoogle Scholar
  10. 10.
    Runtime Team, LaBRI-Inria Futurs: Marcel: A POSIX-compliant thread library for hierarchical multiprocessor machines (2007),
  11. 11.
    Danjean, V., Namyst, R., Russell, R.: Integrating kernel activations in a multithreaded runtime system on Linux. In: Parallel and Distributed Processing. Proc. 4th Workshop on Runtime Systems for Parallel Programming (RTSPP 2000) (2000)Google Scholar
  12. 12.
    Russel, P.: Unreliable guide to hacking the linux kernel (2000)Google Scholar
  13. 13.
    Langendoen, K., Romein, J., Bhoedjang, R., Bal, H.: Integrating polling, interrupts, and thread management. frontiers 00, 13 (1996)Google Scholar
  14. 14.
    Maquelin, O., et al.: Polling watchdog: combining polling and interrupts for efficient message handling. In: ISCA 1996: Proceedings of the 23rd annual international symposium on Computer architecture (1996)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • François Trahay
    • 1
  • Alexandre Denis
    • 1
  • Olivier Aumage
    • 1
  • Raymond Namyst
    • 1
  1. 1.INRIA, LaBRI, Université Bordeaux 1, 351, cours de la Libération, F-33405 TALENCEFrance

Personalised recommendations