Graph-Based Automatic Dynamic Load Balancing for HPC Agent-Based Simulations

  • Claudio MárquezEmail author
  • Eduardo César
  • Joan Sorribes
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9523)


The main problem of Agent-Based Modelling (ABM) simulations in High Performance Computing (HPC) is load imbalance due to a non-uniform distribution of the agents that may generate uneven computation and increase communication overhead, inhibiting the efficiency of the available computing resources. Moreover, the agents’ behaviours can considerably modify the workload at each simulation step thereby affecting the workload progression of the simulation. In order to mitigate such problems, automatic mechanisms for dynamically adjusting the computation and/or communication workload are needed. For this reason, we introduce an Automatic Dynamic Load Balancing (ADLB) strategy to reduce imbalance problems as the simulation proceeds. The ADLB tunes the global simulation workload migrating groups of agents among the processes according to their computation workload and their message connectivity map modelled using a Hypergraph. This Hypergraph is partitioned using the Zoltan Parallel HyperGraph partitioner method (PHG). In addition, to prevent excessive all-to-all communications, the ADLB uses filtering routines to send message groups to specified recipient processes in a simple 3D grid-based structure. Our method has been tested with a biological ABM using the framework Flexible Large-scale Agent Modelling Environment (Flame), obtaining a significant impact on the application performance.


Agent-based simulation Graph partitioning Message filtering Load balancing Performance tuning HPC SPMD Flame 


  1. 1.
    Angiopredict: Predictive genomic biomarkers methods for combination bevacizumab (avastin) therapy in metastatic colorectal cancer ANGIOPREDICT. EU’s Framework Programme Seven (FP7) under contract 306021 (2014).
  2. 2.
    Bithell, M., Macmillan, W.: Escape from the cell: spatially explicit modelling with and without grids. Ecol. Model. 200(1–2), 59–78 (2007)CrossRefGoogle Scholar
  3. 3.
    Coakley, S., Gheorghe, M., Holcombe, M., Chin, S., Worth, D., Greenough, C.: Exploitation of high performance computing in the flame agent-based simulation framework. In: 2012 IEEE 9th International Conference on High Performance Computing and Communication, 2012 IEEE 14th International Conference on Embedded Software and Systems (HPCC-ICESS), pp. 538–545, June 2012Google Scholar
  4. 4.
    Devine, K., Boman, E., Heaphy, R., Bisseling, R., Catalyurek, U.: Parallel hypergraph partitioning for scientific computing. In: 20th International Parallel and Distributed Processing Symposium, IPDPS 2006, p. 10, April 2006Google Scholar
  5. 5.
    Lengauer, T.: Combinatorial Algorithms for Integrated Circuit Layout. Wiley, New York (1990)zbMATHGoogle Scholar
  6. 6.
    Márquez, C., César, E., Sorribes, J.: A load balancing schema for agent-based SPMD applications. In: Parallel and Distributed Processing Techniques and Applications (2013)Google Scholar
  7. 7.
    Márquez, C., César, E., Sorribes, J.: Agent migration in HPC systems using FLAME. In: an Mey, D., Alexander, M., Bientinesi, P., Cannataro, M., Clauss, C., Costan, A., Kecskemeti, G., Morin, C., Ricci, L., Sahuquillo, J., Schulz, M., Scarano, V., Scott, S.L., Weidendorfer, J. (eds.) Euro-Par 2013. LNCS, vol. 8374, pp. 523–532. Springer, Heidelberg (2014) CrossRefGoogle Scholar
  8. 8.
    Márquez, C., César, E., Sorribes, J.: Impact of message filtering on HPC agent-based simulations. In: Proceedings of 28th European Simulation and Modelling Conference, ESM 2014, pp. 65–72. EUROSIS (2014)Google Scholar
  9. 9.
    Parry, H.R., Bithell, M.: Large scale agent-based modelling: a review and guidelines for model scaling. In: Heppenstall, A.J., Crooks, A.T., See, L.M., Batty, M. (eds.) Agent-Based Models of Geographical Systems, pp. 271–308. Springer, Dordrecht (2012)CrossRefGoogle Scholar
  10. 10.
    Petkovic, T., Loncaric, S.: Supercover plane rasterization - a rasterization algorithm for generating supercover plane inside a cube. In: GRAPP (GM/R 2007, pp. 327–332 (2007)Google Scholar
  11. 11.
    Plastino, A., Ribeiro, C.C., Rodriguez, N.: Developing spmd applications with load balancing. Parallel Comput. 29(6), 743–766 (2003). CrossRefGoogle Scholar
  12. 12.
    Solar, R., Suppi, R., Luque, E.: Proximity load balancing for distributed cluster-based individual-oriented fish school simulations. Procedia Comput. Sci. 9, 328–337 (2012). Proceedings of the International Conference on Computational Science, ICCS 2012CrossRefGoogle Scholar
  13. 13.
    Vigueras, G., Lozano, M., Orduña, J.M.: Workload balancing in distributed crowd simulations: the partitioning method. J. Supercomput. 58(2), 261–269 (2011)CrossRefGoogle Scholar
  14. 14.
    Zhang, D., Jiang, C., Li, S.: A fast adaptive load balancing method for parallel particle-based simulations. Simul. Model. Pract. Theor. 17(6), 1032–1042 (2009)CrossRefMathSciNetGoogle Scholar
  15. 15.
    Zheng, G., Meneses, E., Bhatele, A., Kale, L.V.: Hierarchical load balancing for Charm++ applications on large supercomputers. In: Proceedings of the 2010 39th International Conference on Parallel Processing Workshops, ICPPW 2010, pp. 436–444. IEEE Computer Society, Washington, DC (2010)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Claudio Márquez
    • 1
    Email author
  • Eduardo César
    • 1
  • Joan Sorribes
    • 1
  1. 1.Computer Architecture and Operating Systems Department (CAOS)Universitat Autònoma de BarcelonaBellaterraSpain

Personalised recommendations