Advertisement

Simulation and Realistic Workloads to Support the Meta-scheduling of Scientific Workflows

  • Sergio Hernández
  • Javier Fabra
  • Pedro Álvarez
  • Joaquín Ezpeleta
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 256)

Abstract

When heterogeneous computing resources are integrated to create more powerful execution environments, new scheduling strategies are necessary to allocate work units to available resources. In this paper we apply simulation results to schedule the execution of scientific workflows in a resource integration platform. A simulator built upon Alea and GridSim has been implemented to simulate the behaviour of the grid and cluster computing resources integrated in the platform. Simulations are generated using realistic workloads and then analysed by a meta-scheduler to decide the most suitable resource for each workflow task execution. To improve simulation results synthetic workloads are dynamically created considering the current resources state and a set of log-recorded historical executions. The paper also reports the impact of the proposed techniques when experimentally applied to the execution of the Inspiral analysis workflow.

Keywords

Grid Modelling And Simulation Scientific Workflow Workloads Performance Analysis 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Foster, I., Kesselman, C.: The Grid 2: Blueprint for a New Computing Infrastructure. Morgan Kaufmann Publishers Inc., San Francisco (2003)Google Scholar
  2. 2.
    Rahman, M., Ranjan, R., Buyya, R., Benatallah, B.: A taxonomy and survey on autonomic management of applications in grid computing environments. Concurrency and Computation: Practice and Experience 23, 1990–2019 (2011)CrossRefGoogle Scholar
  3. 3.
    Yu, J., Buyya, R.: A taxonomy of scientific workflow systems for grid computing. SIGMOD Record 34, 44–49 (2005)CrossRefGoogle Scholar
  4. 4.
    Kertész, A., Kacsuk, P.: GMBS: A new middleware service for making grids interoperable. Future Generation Computer Systems 26, 542–553 (2010)CrossRefGoogle Scholar
  5. 5.
    Kacsuk, P., Kiss, T., Sipos, G.: Solving the grid interoperability problem by P-GRADE portal at workflow level. Futur. Gener. Comp. Syst. 24, 744–751 (2008)CrossRefGoogle Scholar
  6. 6.
    Hamscher, V., Schwiegelshohn, U., Streit, A., Yahyapour, R.: Evaluation of Job-Scheduling Strategies for Grid Computing. In: Buyya, R., Baker, M. (eds.) GRID 2000. LNCS, vol. 1971, pp. 191–202. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  7. 7.
    Abraham, A., Liu, H., Zhang, W., Chang, T.G.: Scheduling Jobs on Computational Grids Using Fuzzy Particle Swarm Algorithm. In: Gabrys, B., Howlett, R.J., Jain, L.C. (eds.) KES 2006, Part II. LNCS (LNAI), vol. 4252, pp. 500–507. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  8. 8.
    Yu, Z., Shi, W.: An Adaptive Rescheduling Strategy for Grid Workflow Applications. In: IEEE International Parallel and Distributed Processing Symposium, IPDPS 2007, pp. 1–8 (2007)Google Scholar
  9. 9.
    Ludwig, S.A., Moallem, A.: Swarm Intelligence Approaches for Grid Load Balancing. Journal of Grid Computing 9, 279–301 (2011)CrossRefGoogle Scholar
  10. 10.
    Feitelson, D.G.: Workload Modeling for Performance Evaluation. In: Calzarossa, M.C., Tucci, S. (eds.) Performance 2002. LNCS, vol. 2459, pp. 114–141. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  11. 11.
    Iosup, A., Epema, D.H.J.: Grid Computing Workloads. IEEE Internet Computing 15, 19–26 (2011)CrossRefGoogle Scholar
  12. 12.
    Li, H., Groep, D., Wolters, L.: Workload characteristics of a multi-cluster supercomputer. In: Feitelson, D.G., Rudolph, L., Schwiegelshohn, U. (eds.) JSSPP 2004. LNCS, vol. 3277, pp. 176–193. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  13. 13.
    Fabra, J., Hernández, S., Álvarez, P., Ezpeleta, J.: A framework for the flexible deployment of scientific workflows in grid environments. In: Proceedings of the Third International Conference on Cloud Computing, GRIDs, and Virtualization, Cloud Computing 2012, pp. 1–8 (2012)Google Scholar
  14. 14.
    HTCondor Middleware, http://research.cs.wisc.edu/htcondor/ (accessed March 5, 2013)
  15. 15.
    Klusáček, D., Rudová, H.: Alea 2 – Job Scheduling Simulator. In: Proceedings of the 3rd International ICST Conference on Simulation Tools and Techniques, SIMUTools 2010 (2010)Google Scholar
  16. 16.
    Sulistio, A., Cibej, U., Venugopal, S., Robic, B., Buyya, R.: A toolkit for modelling and simulating data Grids: an extension to GridSim. Concurrency and Computation: Practice and Experience 20, 1591–1609 (2008)CrossRefGoogle Scholar
  17. 17.
    Hernández, S., Fabra, J., Álvarez, P., Ezpeleta, J.: A Simulation-based Scheduling Strategy for Scientific Workflows. In: Proceedings of the 2nd International Conference on Simulation and Modeling Methodologies, Technologies and Applications. SIMULTECH 2012, pp. 61–70 (2012)Google Scholar
  18. 18.
    Sargent, R.G.: Verification and validation of simulation models. In: Proceedings of the 2010 Winter Simulation Conference, WSC 2010, pp. 166–183 (2010)Google Scholar
  19. 19.
    gLite Middleware, http://glite.cern.ch/ (accessed March 5, 2013)
  20. 20.
    Taylor, I.J., Deelman, E., Gannon, D.B., Shields, M.: Workflows for e-Science: Scientific Workflows for Grids. Springer-Verlag New York, Inc., Secaucus (2006)Google Scholar
  21. 21.
    Iosup, A., Sonmez, O., Anoep, S., Epema, D.: The performance of bags-of-tasks in large-scale distributed systems. In: Proceedings of the 17th International Symposium on High Performance Distributed Computing, HPDC 2008, pp. 97–108 (2008)Google Scholar
  22. 22.
    Lublin, U., Feitelson, D.G.: The workload on parallel supercomputers: modeling the characteristics of rigid jobs. Journal of Parallel and Distributed Computing 63, 1105–1122 (2003)CrossRefMATHGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Sergio Hernández
    • 1
  • Javier Fabra
    • 1
  • Pedro Álvarez
    • 1
  • Joaquín Ezpeleta
    • 1
  1. 1.Aragón Institute of Engineering Research (I3A), Department of Computer Science and Systems EngineeringUniversity of ZaragozaZaragozaSpain

Personalised recommendations