The Journal of Supercomputing

, Volume 72, Issue 12, pp 4601–4628 | Cite as

An open-source family of tools to reproduce MPI-based workloads in interconnection network simulators

  • Francisco J. Andújar
  • Juan A. Villar
  • Francisco J. Alfaro
  • José L. Sánchez
  • Jesus Escudero-Sahuquillo


Simulation is often used in order to evaluate the behavior and the performance of computing systems. Specifically, in the field of high-performance interconnection networks for HPC clusters the simulation has been extensively considered to verify and validate network operation models and to evaluate their performance. Nevertheless, experiments conducted to evaluate network performance using simulation tools should be fed with realistic network traffic from real benchmarks and/or applications. This approach has grown in popularity because it allows to evaluate the simulation model under realistic traffic situations. In this paper, we propose a family of tools for modeling realistic workloads which capture the behavior of MPI applications into self-related traces called VEF traces. The main novelty of this approach is that it replays the MPI collective operations with their corresponding messages, offering an MPI message-based task simulation framework. The proposed framework neither provides a network simulator nor depends on any specific simulation platform. Besides, this framework allows us to use the generated traces by any third-party network simulator working at message level.


HPC Interconnection network Message passing interface Extrae VEF traces Self-related traces Application traces Traffic model 



This work has been jointly supported by the MINECO and European Commission (FEDER funds) under the project TIN2015-66972-C5-2-R, and by Junta de Comunidades de Castilla-La Mancha under the Project PEII-2014-028-P. Francisco J. Andújar is also funded by the Spanish Ministry of Science and Innovation MICINN under FPU grant AP2010-4680 and Jesus Escudero-Sahuquillo has been funded by the Spanish MINECO under the postdoctoral grant FPDI-2013-18787 until November 2015 and, from that date, he has been funded by the University of Castilla-La Mancha (UCLM) and the European Commission (FSE funds), with a contract for accessing the Spanish System of Science, Technology and Innovation, for the implementation of the UCLM research program (UCLM resolution date: 31/07/2014).


  1. 1.
    Anderson T, Owicki S, Saxe J, Thacker C (1993) High-speed switch scheduling for local-area networks. ACM Trans Comput Syst 11:319–352CrossRefGoogle Scholar
  2. 2.
    Andújar FJ, Villar JA, Sánchez JL, Alfaro FJ, Duato J (2015) N-dimensional twin torus topology. IEEE Trans Comput 64(10):2847–2861MathSciNetCrossRefGoogle Scholar
  3. 3.
    Andújar FJ, Villar JA, Sánchez JL, Alfaro FJ, Escudero-Sahuquillo J (2015) VEF traces: a framework for modelling MPI traffic in interconnection network simulators. The 1st IEEE International Workshop on High-Performance Interconnection Networks in the Exascale and Big-Data Era (HiPINEB 2015), co-located with 2015 IEEE International Conference on Cluster Computing (CLUSTER 2015), Chicago, IL, USA, pp 841–848Google Scholar
  4. 4.
    Birke R, Rodriguez G, Minkenberg C (2012) Towards massively parallel simulations of massively parallel high-performance computing systems. In: Proceedings of the 5th International Conference on Simulation Tools and Techniques, Brussels, Belgium, pp 291–298Google Scholar
  5. 5.
    Casanova H, Desprez F, Markomanolis GS, Suter F (2015) Simulation of MPI applications with time-independent traces. Concurr Comput Pract Exp 27(5):1145–1168CrossRefGoogle Scholar
  6. 6.
    DIMEMAS (2015) Dimemas: predict parallel performance using a single cpu machine. Accessed May 2016
  7. 7.
    Duato J, Yalamanchili S, Ni L (2003) Interconnection networks. An engineering approach. Morgan Kaufmann Publishers Inc, San Francisco, CA, USAGoogle Scholar
  8. 8.
    EXTRAE (2015) Homepage of the library Extrae. Accessed May 2016
  9. 9.
    Gabriel E et al (2004) Open MPI: goals, concept, and design of a next generation MPI implementation. In: Proceedings of the 11th European PVM/MPI Users’ Group Meeting. Budapest, Hungary, pp 97–104Google Scholar
  10. 10.
    GALGO (2015) GALGO—Albacete Research Institute of Informatics Supercomputer Center. Accessed May 2016
  11. 11.
    GRAPH500 (2015) Homepage of Graph500. Accessed May 2016
  12. 12.
    Janssen CL, Adalsteinsson H, Cranford S, Kenny JP, Pinar A, Evensky DA, Mayo J (2010) A simulator for large-scale parallel computer architectures. J Distrib Syst Technol 1(2):57–73CrossRefGoogle Scholar
  13. 13.
    Karol M, Hluchyj M (1998) Queuing in high-performance packet-switching. IEEE J Sel Areas 1:1587–1597Google Scholar
  14. 14.
    Karrels E, Lusk E (1994) Performance analysis of MPI program. In: Proceedings of the workshop on Environments and Tools for Parallel Scientific Computing, pp 195–200Google Scholar
  15. 15.
    Miguel-Alonso J, Navaridas J, Ridruejo F (2008) Interconnection network simulation using traces of mpi applications. Tech. Rep. EHU-KAT-IK-01-08, Department of Computer Architecture and Technology, The Universiy of the Basque Country, SpainGoogle Scholar
  16. 16.
    Miguel-Alonso J, Navaridas J, Ridruejo F (2009) Interconnection network simulation using traces of MPI applications. Int J Parallel Program 37(2):153–174CrossRefzbMATHGoogle Scholar
  17. 17.
    Nagel WE, Arnold A, Weber M, Hoppe HC, Solchenbach K (1996) Vampir: visualization and analysis of mpi resources. Supercomputer 12:69–80Google Scholar
  18. 18.
    Núñez A, Fernández J, García J, Carretero J (2008) New techniques for simulating high performance MPI applications on large storage networks. In: Proceedings of the IEEE International Conference on Cluster Computing, Tsukuba, Japan, pp 444–452Google Scholar
  19. 19.
    Penoff B, Wagner A, Tüxen M, Rüngeler I (2009) MPI-NeTSim: a network simulation module for MPI. In: Proceedings of the 15th International Conference on Parallel and Distributed Systems. IEEE Computer Society, Washington, DC, USA, pp 464–471Google Scholar
  20. 20.
    Pillet V, Labarta J, Cortes T, Girona S (1995) Paraver: a tool to visualize and analyze parallel code. Tech. Rep. RR-95/03, Departament D’arquitectura De Computadors. Universitat Politècnica de Catalunya, SpainGoogle Scholar
  21. 21.
    Riesen R (2006) A hybrid MPI simulator. In: Proceedings of the IEEE International Conference on Cluster Computing, Barcelona, Spain, pp 1–9Google Scholar
  22. 22.
    Shende SS, Malony AD (2006) The Tau parallel performance system. Int J High Perform Comput Appl 20(2):287–311CrossRefGoogle Scholar
  23. 23.
    Snir M, Otto S, Huss-Lederman S, Walker D, Dongarra J (1998) MPI-the complete reference, vol 1. The MPI Core, 2nd edn. MIT Press, CambridgeGoogle Scholar
  24. 24.
    SST (2015) The structural simulation toolkit. Accessed May 2016
  25. 25.
    Tikir M, Laurenzano M, Carrington L, Snavely A (2009) PSINS: an open source event tracer and execution simulator for MPI applications. In: Euro-Par 2009 Parallel Processing, Lecture Notes in Computer Science, vol 5704. Springer, Berlin Heidelberg, pp 135–148Google Scholar
  26. 26.
    Triviño F, Andújar F, Alfaro F, Sánchez J, Ros A (2011) Self-related traces: an alternative to full-system simulation for NoCs. In: Proceedings of the International Conference on High Performance Computing and Simulation, Istanbul, Turkey, pp 819–824Google Scholar
  27. 27.
    VEF (2015) VEF traces homepage. Accessed May 2016
  28. 28.
    Villar JA, Andújar FJ, Sánchez JL, Alfaro FJ, Gámez JA, Duato J (2013) Obtaining the optimal configuration of high-radix combined switches. J Parallel Distrib Comput 73(9):1239–1250. doi: 10.1016/j.jpdc.2013.04.009 CrossRefGoogle Scholar
  29. 29.
    Yebenes P, Escudero-Sahuquillo J, Requena CG, García PJ, Quiles FJ, Duato J (2013) BBQ: a straightforward queuing scheme to reduce HoL-blocking in high-performance hybrid networks. In: Proceedings of the 19th International Conference Euro-Par Parallel Processing, Aachen, Germany, pp 699–712Google Scholar
  30. 30.
    Zheng G, Kakulapati G, Kale L (2004) BigSim: a parallel simulator for performance prediction of extremely large parallel machines. In: Proceedings of the 18th International Parallel and Distributed Processing Symposium, Santa Fe, New Mexico, USA, p 78Google Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  • Francisco J. Andújar
    • 1
  • Juan A. Villar
    • 1
  • Francisco J. Alfaro
    • 1
  • José L. Sánchez
    • 1
  • Jesus Escudero-Sahuquillo
    • 2
  1. 1.Computing Systems DepartmentUniversity of Castilla-La ManchaAlbaceteSpain
  2. 2.Department of Computing EngineeringTechnical University of ValenciaValenciaSpain

Personalised recommendations