Abstract
Large-scale federated environments have emerged to meet the requirements of increasingly demanding scientific applications. However, the seemingly unlimited availability of computing resources and heterogeneity turns the scheduling into an NP-hard problem. Unlike exhaustive algorithms and deterministic heuristics, evolutionary algorithms have been shown appropriate for large-scheduling problems, obtaining near optimal solutions in a reasonable time. In the present work, we propose a Genetic Algorithm (GA) for scheduling job-packages of parallel task in resource federated environments. The main goal of the proposal is to determine the job schedule and package allocation to improve the application performance and system throughput. To address such a complex infrastructure, the GA is provided with knowledge based on slowdown predictions for the application runtime, obtained by considering heterogeneity and bandwidth issues. The proposed GA algorithm was tuned and evaluated using real workload traces and the results compared with a range of well-known heuristics in the literature.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Braun, T.D., Siegel, H.J., Beck, N., Bölöni, L.L., Maheswaran, M.-C., Reuther, A.I., Robertson, J.P., Theys, M.D., Yao, B., Hensgen, D., Freund, R.F.: A comparison of eleven static heuristics for mapping a class of independent tasks onto heterogeneous distributed computing systems. Journal of Parallel and Distributed Computing 61(6), 810–837 (2001)
Kolodziej, J., Xhafa, F.: Enhancing the genetic-based scheduling in computational grids by a structured hierarchical population. Future Generation Computer Systems 27(8), 1035–1046 (2011)
Mathiyalagan, P., Suriya, S., Sivanandam, S.N.: Hybrid enhanced ant colony algorithm and enhanced bee colony algorithm for grid scheduling. Int. J. Grid Util. Comput. 2(1), 45–58 (2011)
Blanco, H., Llados, J., Guirado, F., Lerida, J.L.: Ordering and allocating parallel jobs on multi-cluster systems. In: CMMSE, pp. 196–206 (2012)
Ernemann, C., Hamscher, V., Schwiegelshohn, U., Yahyapour, R., Streit, A.: On advantages of grid computing for parallel job scheduling. In: CCGRID, pp. 39–39. IEEE (2002)
Bucur, A.I.D., Epema, D.H.J.: Scheduling policies for processor coallocation in multicluster systems. IEEE Transactions on Parallel and Distributed Systems 18(7), 958–972 (2007)
Blanco, H., Lerida, J.L., Cores, F., Guirado, F.: Multiple job co-allocation strategy for heterogeneous multi-cluster systems based on linear programming. The Journal of Supercomputing 58(3), 394–402 (2011)
Jones, W.M., Ligon III, W.B., Pang, L.W., Stanzione Jr., D.C.: Characterization of bandwidth-aware meta-schedulers for co-allocating jobs across multiple clusters. The Journal of Supercomputing 34(2), 135–163 (2005)
Liu, D., Han, N.: Co-scheduling deadline-sensitive applications in large-scale grid systems. International Journal of Future Generation Communication & Networking 7(3), 49–60 (2014)
Mohamed, H.H., Epema, D.H.J.: An evaluation of the close-to-files processor and data co-allocation policy in multiclusters. In: IEEE CLUSTER, pp. 287–298 (2004)
Finger, M., Capistrano, G., Bezerra, C., Conde, D.R.: Resource use pattern analysis for predicting resource availability in opportunistic grids. Concurrency and Computation: Practice and Experience 22(3), 295–313 (2010)
Wang, C.-M., Chen, H.-M., Hsu, C.-C., Lee, J.: Dynamic resource selection heuristics for a non-reserved bidding-based grid environment. Future Generation Computer Systems 26(2), 183–197 (2010)
Tsafrir, D., Etsion, Y., Feitelson, D.G.: Backfilling using system-generated predictions rather than user runtime estimates. IEEE Transactions on Parallel and Distributed Systems 18(6), 789–803 (2007)
Shmueli, E., Feitelson, D.G.: Backfilling with lookahead to optimize the packing of parallel jobs. Journal of Parallel and Distributed Computing 65(9), 1090–1107 (2005)
Blanco, H., Guirado, F., Lerida, J.L., Albornoz, V.M.: Mip model scheduling for bsp parallel applications on multi-cluster environments. In: 3PGCIC, pp. 12–18. IEEE (2012)
Naik, V.K., Liu, C., Yang, L., Wagner, J.: Online resource matching for heterogeneous grid environments. In: CCGRID, pp. 607–614 (2005)
Carretero, J., Xhafa, F., Abraham, A.: Genetic algorithm based schedulers for grid computing systems. International Journal of Innovative Computing, Information and Control 3(6), 1–19 (2007)
Zomaya, A.Y., Teh, Y.-H.: Observations on using genetic algorithms for dynamic load-balancing. IEEE Transactions on Parallel and Distributed Systems 12(9), 899–911 (2001)
Garg, S.K.: Gridsim simulation framework (2009). http://www.buyya.com/gridsim
Feitelson, D.: Parallel workloads archive (2005). http://www.cs.huji.ac.il/labs/parallel/workload
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Gabaldon, E., Lerida, J.L., Guirado, F., Planes, J. (2015). Slowdown-Guided Genetic Algorithm for Job Scheduling in Federated Environments. In: Vinh, P., Vassev, E., Hinchey, M. (eds) Nature of Computation and Communication. ICTCC 2014. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 144. Springer, Cham. https://doi.org/10.1007/978-3-319-15392-6_18
Download citation
DOI: https://doi.org/10.1007/978-3-319-15392-6_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-15391-9
Online ISBN: 978-3-319-15392-6
eBook Packages: Computer ScienceComputer Science (R0)