FLEX-MPI: An MPI Extension for Supporting Dynamic Load Balancing on Heterogeneous Non-dedicated Systems

  • Gonzalo Martín
  • Maria-Cristina Marinescu
  • David E. Singh
  • Jesús Carretero
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8097)


This paper introduces FLEX-MPI, a novel runtime approach for the dynamic load balancing of MPI-based SPMD applications running on heterogeneous platforms in the presence of dynamic external loads. To effectively balance the workload, FLEX-MPI monitors the actual performance of applications via hardware counters and the MPI profiling interface—with a negligible overhead and minimal code modifications. Our results show that by using this approach the execution time of an application may be significantly reduced.


Dynamic load balancing distributed computing heterogeneous systems hardware counters 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bailey, D., et al.: The NAS parallel benchmarks summary and preliminary results. In: Proceedings of the 1991 ACM/IEEE Conference on Supercomputing, pp. 158–165. IEEE (1991)Google Scholar
  2. 2.
    Martín, G., Marinescu, M., Singh, D., Carretero, J.: Leveraging social networks for understanding the evolution of epidemics. BMC Syst. Biol. 5(suppl. 3) (2011)Google Scholar
  3. 3.
    Xu, C., Lau, F.: Load balancing in parallel computers: theory and practice. Kluwer Academic Publishers (1997)Google Scholar
  4. 4.
    Beltran, M., Guzman, A., Bosque, J.: Dealing with heterogeneity in load balancing algorithms. In: ISPDC 2006, pp. 123–132. IEEE (2006)Google Scholar
  5. 5.
    Belikov, E., Loidl, H., Michaelson, G., Trinder, P.: Architecture-aware cost modelling for parallel performance portability. In: Kolloquium Programmiersprachen und Grundlagen der Programmierung 5Google Scholar
  6. 6.
    Galindo, I., Almeida, F., Badía-Contelles, J.M.: Dynamic load balancing on dedicated heterogeneous systems. In: Lastovetsky, A., Kechadi, T., Dongarra, J. (eds.) EuroPVM/MPI 2008. LNCS, vol. 5205, pp. 64–74. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  7. 7.
    Martínez, J., Almeida, F., Garzón, E., Acosta, A., Blanco, V.: Adaptive load balancing of iterative computation on heterogeneous nondedicated systems. The Journal of Supercomputing 58(3), 385–393 (2011)CrossRefGoogle Scholar
  8. 8.
    Weatherly, D., Lowenthal, D., Nakazawa, M., Lowenthal, F.: Dyn-MPI: Supporting mpi on medium-scale, non-dedicated clusters. Journal of Parallel and Distributed Computing 66(6), 822–838 (2006)zbMATHCrossRefGoogle Scholar
  9. 9.
    Bohn, C., Lamont, G.: Load balancing for heterogeneous clusters of PCs. Future Generation Computer Systems 18(3), 389–400 (2002)CrossRefGoogle Scholar
  10. 10.
    Lastovetsky, A., Reddy, R.: HeteroMPI: Towards a message-passing library for heterogeneous networks of computers. Journal of Parallel and Distributed Computing 66(2), 197–220 (2006)zbMATHCrossRefGoogle Scholar
  11. 11.
    Huang, C., Lawlor, O., Kale, L.: Adaptive MPI. In: Rauchwerger, L. (ed.) LCPC 2003. LNCS, vol. 2958, pp. 306–322. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  12. 12.
    Dongarra, J., Malony, A., Moore, S., Mucci, P., Shende, S.: Performance instrumentation and measurement for terascale systems. In: Sloot, P.M.A., Abramson, D., Bogdanov, A.V., Gorbachev, Y.E., Dongarra, J., Zomaya, A.Y. (eds.) ICCS 2003, Part IV. LNCS, vol. 2660, pp. 53–62. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  13. 13.
    Martín, G., Marinescu, M., Singh, D., Carretero, J.: FLEX-MPI - Technical Report. Technical report, Universidad Carlos III de Madrid (2012),
  14. 14.
    Mucci, P., Browne, S., Deane, C., Ho, G.: PAPI: A portable interface to hardware performance counters. In: Proceedings of the Department of Defense HPCMP Users Group Conference, 7–10 (1999)Google Scholar
  15. 15.
    Davis, T.A., Hu, Y.: The University of Florida sparse matrix collection. ACM Trans. Math. Softw. 38(1), 1:1–1:25 (2011)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Gonzalo Martín
    • 1
  • Maria-Cristina Marinescu
    • 2
  • David E. Singh
    • 1
  • Jesús Carretero
    • 1
  1. 1.Universidad Carlos III de MadridLeganésSpain
  2. 2.Barcelona Supercomputing CenterBarcelonaSpain

Personalised recommendations