Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Adaptive load balancing of iterative computation on heterogeneous nondedicated systems

  • 84 Accesses

  • 10 Citations

Abstract

Dynamic load balancing in heterogeneous systems is a fundamental research topic in parallel computing due to the high availability of such systems. The efficient utilization of the heterogeneous resources can significantly enhance the performance of the parallel system. At the same time, adapting parallel codes to state-of-the-art parallel computers composed of heterogeneous multinode–multicore processors becomes a very hard task because parallel codes are highly dependent on the parallel architectures. That means that applications must be tailored requiring a great deal of programming effort. We have developed the ALBIC (Adaptive Load Balancing of Iterative Computation) system that allows for the dynamic load balancing of iterative codes in heterogeneous dedicated and nondedicated Linux based systems. In order to validate the system several parallel codes have been analyzed in different scenarios. The results show that the ALBIC approach achieves better performance than the other proposal. This lightweighted library eases porting homogeneous parallel codes to heterogeneous platforms, since the code intrusion is low and the programming effort is quite reduced.

This is a preview of subscription content, log in to check access.

References

  1. 1.

    Aliaga JI, Almeida F, Badía-Contelles JM, Barrachina-Mir S, Blanco V, Castillo MI, Dorta U, Mayo R, Quintana-Ortí ES, Quintana-Ortí G, Rodríguez C, de Sande F (2004) Parallelization of the gnu scientific library on heterogeneous systems. In: ISPDC/HeteroPar. IEEE Computer Society, Los Alamitos, pp 338–345

  2. 2.

    Almeida F, González D, Moreno LM (2006) The master-slave paradigm on heterogeneous systems: a dynamic programming approach for the optimal mapping. J Syst Archit 52(2):105–116

  3. 3.

    Beltrán M, Guzmán A, Bosque JL (2006) Dealing with heterogeneity in load balancing algorithms. In: ISPDC. IEEE Computer Society, Los Alamitos, pp 123–132

  4. 4.

    Bosque JL, Marcos D Gil, Pastor L (2004) Dynamic load balancing in heterogeneous clusters. In: Hamza MH (ed) Parallel and distributed computing and networks. IASTED/ACTA Press, Anaheim, pp 37–42

  5. 5.

    Bovet D, Cesati M (2002) Understanding the linux kernel, 2nd edn. O’Reilly & Associates, Sebastopol

  6. 6.

    Chen Z, Yang M, Francia GA III, Dongarra J (2007) Self adaptive application level fault tolerance for parallel and distributed computing. In: IPDPS. IEEE Press, New York, pp 1–8

  7. 7.

    Cuenca J, Giménez D, Martinez JP (2005) Heuristics for work distribution of a homogeneous parallel dynamic programming scheme on heterogeneous systems. Parallel Comput 31(7):711–735

  8. 8.

    Dongarra J, Bosilca G, Chen Z, Eijkhout V, Fagg GE, Fuentes E, Langou J, Luszczek P, Pjesivac-Grbovic J, Seymour K, You H, Vadhiyar SS (2006) Self-adapting numerical software (sans) effort. IBM J Res Dev 50(2–3):223–238

  9. 9.

    Galindo I, Almeida F, Blanco V, Badía JM Dynamic load balancing on dedicated heterogeneous system. In: Grosspietsch E, Klöckner K (eds) 16th euromicro international conference on parallel, distributed and network-based processing, vol SEA-SR-18, Toulouse, France, February 2008. Institute for Systems Engineering and Automation

  10. 10.

    HeteroMPI: Mpi extension for heterogeneous networks of computers. http://hcl.ucd.ie/Projects/HeteroMPI

  11. 11.

    Huang C, Lawlor O, Kale L (2003) Adaptive MPI. In: 16th international workshop on languages and compilers for parallel computing (LCPC). LNCS, vol 2958, pp 306–322

  12. 12.

    Kalinov A (2006) Scalability of heterogeneous parallel systems. Program Comput Softw 32(1):1–7

  13. 13.

    Kalinov A, Lastovetsky AL, Robert Y (2005) Heterogeneous computing. Parallel Comput 31(7):649–652

  14. 14.

    Lastovetsky A, Reddy R (2006) HeteroMPI: Towards a message-passing library for heterogeneous networks of computers. J Parallel Distrib Comput 66:197–220

  15. 15.

    Martíne JA, Garzón EM, Plaza A, García I (2009) Automatic tuning of iterative computation on heterogeneous multiprocessors with ADITHE. J Supercomput. doi:10.1007/s11227-009-0350-1

  16. 16.

    mpC: parallel programming language for heterogeneous networks of computers. http://hcl.ucd.ie/Projects/mpC

  17. 17.

    Weatherly D, Lowenthal DK, Nakazawa M, Lowenthal F (2006) Dyn-MPI: Supporting MPI on medium-scale, non-dedicated clusters. J Parallel Distrib Comput 66(6):822–838

Download references

Author information

Correspondence to E. M. Garzón.

Additional information

This work has been supported by the EC (FEDER), the Spanish Ministry of Science and Innovation with the I+D+I TIN2008-01117 and TIN2008-06570-C04 contracts; Junta de Andalucia with P08-TIC-3518, P10-TIC-6002 contracts, and Canary Government with SolSubC200801000307 contract and developed in the framework of the network (CAPAP-H) TIN2009-08058-E.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Martínez, J.A., Almeida, F., Garzón, E.M. et al. Adaptive load balancing of iterative computation on heterogeneous nondedicated systems. J Supercomput 58, 385–393 (2011). https://doi.org/10.1007/s11227-011-0595-3

Download citation

Keywords

  • Heterogeneous system
  • Dynamic load balancing