Abstract
When heterogeneous computing resources are integrated to create more powerful execution environments, new scheduling strategies are necessary to allocate work units to available resources. In this paper we apply simulation results to schedule the execution of scientific workflows in a resource integration platform. A simulator built upon Alea and GridSim has been implemented to simulate the behaviour of the grid and cluster computing resources integrated in the platform. Simulations are generated using realistic workloads and then analysed by a meta-scheduler to decide the most suitable resource for each workflow task execution. To improve simulation results synthetic workloads are dynamically created considering the current resources state and a set of log-recorded historical executions. The paper also reports the impact of the proposed techniques when experimentally applied to the execution of the Inspiral analysis workflow.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Foster, I., Kesselman, C.: The Grid 2: Blueprint for a New Computing Infrastructure. Morgan Kaufmann Publishers Inc., San Francisco (2003)
Rahman, M., Ranjan, R., Buyya, R., Benatallah, B.: A taxonomy and survey on autonomic management of applications in grid computing environments. Concurrency and Computation: Practice and Experience 23, 1990–2019 (2011)
Yu, J., Buyya, R.: A taxonomy of scientific workflow systems for grid computing. SIGMOD Record 34, 44–49 (2005)
Kertész, A., Kacsuk, P.: GMBS: A new middleware service for making grids interoperable. Future Generation Computer Systems 26, 542–553 (2010)
Kacsuk, P., Kiss, T., Sipos, G.: Solving the grid interoperability problem by P-GRADE portal at workflow level. Futur. Gener. Comp. Syst. 24, 744–751 (2008)
Hamscher, V., Schwiegelshohn, U., Streit, A., Yahyapour, R.: Evaluation of Job-Scheduling Strategies for Grid Computing. In: Buyya, R., Baker, M. (eds.) GRID 2000. LNCS, vol. 1971, pp. 191–202. Springer, Heidelberg (2000)
Abraham, A., Liu, H., Zhang, W., Chang, T.G.: Scheduling Jobs on Computational Grids Using Fuzzy Particle Swarm Algorithm. In: Gabrys, B., Howlett, R.J., Jain, L.C. (eds.) KES 2006, Part II. LNCS (LNAI), vol. 4252, pp. 500–507. Springer, Heidelberg (2006)
Yu, Z., Shi, W.: An Adaptive Rescheduling Strategy for Grid Workflow Applications. In: IEEE International Parallel and Distributed Processing Symposium, IPDPS 2007, pp. 1–8 (2007)
Ludwig, S.A., Moallem, A.: Swarm Intelligence Approaches for Grid Load Balancing. Journal of Grid Computing 9, 279–301 (2011)
Feitelson, D.G.: Workload Modeling for Performance Evaluation. In: Calzarossa, M.C., Tucci, S. (eds.) Performance 2002. LNCS, vol. 2459, pp. 114–141. Springer, Heidelberg (2002)
Iosup, A., Epema, D.H.J.: Grid Computing Workloads. IEEE Internet Computing 15, 19–26 (2011)
Li, H., Groep, D., Wolters, L.: Workload characteristics of a multi-cluster supercomputer. In: Feitelson, D.G., Rudolph, L., Schwiegelshohn, U. (eds.) JSSPP 2004. LNCS, vol. 3277, pp. 176–193. Springer, Heidelberg (2005)
Fabra, J., Hernández, S., Álvarez, P., Ezpeleta, J.: A framework for the flexible deployment of scientific workflows in grid environments. In: Proceedings of the Third International Conference on Cloud Computing, GRIDs, and Virtualization, Cloud Computing 2012, pp. 1–8 (2012)
HTCondor Middleware, http://research.cs.wisc.edu/htcondor/ (accessed March 5, 2013)
Klusáček, D., Rudová, H.: Alea 2 – Job Scheduling Simulator. In: Proceedings of the 3rd International ICST Conference on Simulation Tools and Techniques, SIMUTools 2010 (2010)
Sulistio, A., Cibej, U., Venugopal, S., Robic, B., Buyya, R.: A toolkit for modelling and simulating data Grids: an extension to GridSim. Concurrency and Computation: Practice and Experience 20, 1591–1609 (2008)
Hernández, S., Fabra, J., Álvarez, P., Ezpeleta, J.: A Simulation-based Scheduling Strategy for Scientific Workflows. In: Proceedings of the 2nd International Conference on Simulation and Modeling Methodologies, Technologies and Applications. SIMULTECH 2012, pp. 61–70 (2012)
Sargent, R.G.: Verification and validation of simulation models. In: Proceedings of the 2010 Winter Simulation Conference, WSC 2010, pp. 166–183 (2010)
gLite Middleware, http://glite.cern.ch/ (accessed March 5, 2013)
Taylor, I.J., Deelman, E., Gannon, D.B., Shields, M.: Workflows for e-Science: Scientific Workflows for Grids. Springer-Verlag New York, Inc., Secaucus (2006)
Iosup, A., Sonmez, O., Anoep, S., Epema, D.: The performance of bags-of-tasks in large-scale distributed systems. In: Proceedings of the 17th International Symposium on High Performance Distributed Computing, HPDC 2008, pp. 97–108 (2008)
Lublin, U., Feitelson, D.G.: The workload on parallel supercomputers: modeling the characteristics of rigid jobs. Journal of Parallel and Distributed Computing 63, 1105–1122 (2003)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Hernández, S., Fabra, J., Álvarez, P., Ezpeleta, J. (2014). Simulation and Realistic Workloads to Support the Meta-scheduling of Scientific Workflows. In: Obaidat, M., Filipe, J., Kacprzyk, J., Pina, N. (eds) Simulation and Modeling Methodologies, Technologies and Applications. Advances in Intelligent Systems and Computing, vol 256. Springer, Cham. https://doi.org/10.1007/978-3-319-03581-9_11
Download citation
DOI: https://doi.org/10.1007/978-3-319-03581-9_11
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-03580-2
Online ISBN: 978-3-319-03581-9
eBook Packages: EngineeringEngineering (R0)