Advertisement

The Multi-level Adaptive Approach for Efficient Execution of Multi-scale Distributed Applications with Dynamic Workload

  • Denis NasonovEmail author
  • Nikolay Butakov
  • Michael Melnik
  • Alexandr Visheratin
  • Alexey Linev
  • Pavel Shvets
  • Sergey Sobolev
  • Ksenia Mukhina
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 965)

Abstract

Today advanced research is based on complex simulations which require a lot of computational resources that usually are organized in a very complicated way from technical part of the view. It means that a scientist from physics, biology or even sociology should struggle with all technical issues on the way of building distributed multi-scale application supported by a stack of specific technologies on high-performance clusters. As the result, created applications have partly implemented logic and are extremely inefficient in execution. In this paper, we present an approach which takes away the user from the necessity to care about an efficient resolving of imbalance of computations being performed in different processes and on different scales of his application. The efficient balance of internal workload in distributed and multi-scale applications may be achieved by introducing: a special multi-level model; a contract (or domain-specific language) to formulate the application in terms of this model; and a scheduler which operates on top of that model. The multi-level model consists of computing routines, computational resources and executed processes, determines a mapping between them and serves as a mean to evaluate the resulting performance of the whole application and its individual parts. The contract corresponds to unification interface of application integration in the proposed framework while the scheduling algorithm optimizes the execution process taking into consideration the main computational environment aspects.

Keywords

Multi-scale applications Distributed computing HPC Optimization Multi-agent modeling MPI 

References

  1. 1.
    Liu, L., Zhang, M., Buyya, R., Fan, Q.: Deadline-constrained coevolutionary genetic algorithm for scientific workflow scheduling in cloud computing. Concurr. Comput. Pract. Exp. 29(5) (2017)Google Scholar
  2. 2.
    Zhou, N., Qi, D., Wang, X., Zheng, Z., Lin, W.: A list scheduling algorithm for heterogeneous systems based on a critical node cost table and pessimistic cost table. Concurr. Comput. Pract. Exp. 29(5) (2017)Google Scholar
  3. 3.
    Visheratin, A.A., Melnik, M., Nasonov, D.: Dynamic resources configuration for coevolutionary scheduling of scientific workflows in cloud environment. In: Pérez García, H., Alfonso-Cendón, J., Sánchez González, L., Quintián, H., Corchado, E. (eds.) SOCO/CISIS/ICEUTE -2017. AISC, vol. 649, pp. 13–23. Springer, Cham (2018).  https://doi.org/10.1007/978-3-319-67180-2_2CrossRefGoogle Scholar
  4. 4.
    Visheratin, A.A., Melnik, M., Nasonov, D.: Automatic workflow scheduling tuning for distributed processing systems. Procedia Comput. Sci. 101, 388–397 (2016)CrossRefGoogle Scholar
  5. 5.
    Chen, H., Zhu, X., Qiu, D., Guo, H., Yang, L.T., Lu, P.: EONS: minimizing energy consumption for executing real-time workflows in virtualized cloud data centers. In: 2016 45th International Conference on Parallel Processing Workshops (ICPPW), pp. 385–392. IEEE, August 2016Google Scholar
  6. 6.
    Wang, Y., Shi, W., Kent, K.B.: On optimal scheduling algorithms for well-structured workflows in the cloud with budget and deadline constraints. Parallel Proc. Lett. 26(02) (2016).  https://doi.org/10.1142/S0129626416500092
  7. 7.
    Balis, B.: HyperFlow: a model of computation, programming approach and enactment engine for complex distributed workflows. Future Gener. Comput. Syst. 55, 147–162 (2016)CrossRefGoogle Scholar
  8. 8.
    Zenmyo, T., Iijima, S., Fukuda, I.: Managing a complicated workflow based on dataflow-based workflow scheduler. In: 2016 IEEE International Conference on Big Data (Big Data), pp. 1658–1663. IEEE, December 2016Google Scholar
  9. 9.
    Borgdorff, J., et al.: Performance of distributed multiscale simulations. Phil. Trans. R. Soc. A 372(2021) (2014).  https://doi.org/10.1098/rsta.2013.0407
  10. 10.
    Lu, S., et al.: A framework for cloud-based large-scale data analytics and visualization: case study on multiscale climate data. In: 2011 IEEE Third International Conference on Cloud Computing Technology and Science (CloudCom), pp. 618–622. IEEE, November 2011Google Scholar
  11. 11.
    da Silva, R.F., Filgueira, R., Pietri, I., Jiang, M., Sakellariou, R., Deelman, E.: A characterization of workflow management systems for extreme-scale applications. Future Gener. Comput. Syst. 75, 228–238 (2017)CrossRefGoogle Scholar
  12. 12.
    Belgacem, M.B., Chopard, B.: MUSCLE-HPC: a new high performance API to couple multiscale parallel applications. Future Gener. Comput. Syst. 67, 72–82 (2017)CrossRefGoogle Scholar
  13. 13.
    Alowayyed, S., Groen, D., Coveney, P.V., Hoekstra, A.G.: Multiscale computing in the exascale era. J. Comput. Sci. 22, 15–25 (2017)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Denis Nasonov
    • 1
    Email author
  • Nikolay Butakov
    • 1
  • Michael Melnik
    • 1
  • Alexandr Visheratin
    • 1
  • Alexey Linev
    • 2
  • Pavel Shvets
    • 3
  • Sergey Sobolev
    • 4
  • Ksenia Mukhina
    • 1
  1. 1.ITMO UniversitySaint-PetersburgRussia
  2. 2.Lobachevsky State University of Nizhni NovgorodNizhny NovgorodRussia
  3. 3.Research Computing Center of Moscow State UniversityMoscowRussia
  4. 4.Moscow State UniversityMoscowRussia

Personalised recommendations