Skip to main content
SpringerLink
Log in

On the use of meta-heuristics to increase the efficiency of online grid workflow scheduling algorithms

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

The competitiveness of online algorithms is measured based on the correctness of the results produced and processing time efficiency. Traditionally evolutionary algorithms are not favored in online paradigms because of the large number of iterations involved in the algorithm which translates directly into processing time overhead. In this paper we describe MARS (Management Architecture for Resource Services) online scheduling algorithm which uses Simulated Annealing and concepts from Tabu Search to drastically decrease the processing time of the algorithm. The paper outlines the concepts behind MARS, the components involved and scheduling methodology used. In addition we also identify the time consuming bottlenecks in the performance of the system and how evolutionary algorithms help us soar past them.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Wu, A.S., Yu, H., Jin, S., Lin, K.-C., Schiavone, G.: An incremental genetic algorithm approach to multiprocessor scheduling. IEEE Trans. Parallel Distributed Syst. 15(9), 824–834 (2004)

    Article  Google Scholar 

  2. Braun, T., Siegel, H.J., Beck, N., Boloni, L.L., Maheswaran, M., Reuther, A.I., Robertson, J.P., Theys, M.D., Yao, B., Hengsen, D., Freund, R.F.: A comparison study of static mapping heuristics for a class of meta-tasks on heterogeneous computing systems. In: Proceedings of the International Heterogenous Computing Workshop (HCW, 99), pp. 15–29, April 1999

  3. Aziz, A., El-Rewini, H.: Grid resource allocation and task scheduling for resource intensive applications. In: PDM Workshop, held in Conjunction with ICPP, Aug 14, 2006

  4. Frey, J., Tannenbaum, T., Foster, I., Livny, M., Tuecke, S.: Condor-G: a computation management agent for multi-institutional grids. J. Cluster Comput. 5, 237–246 (2002)

    Article  Google Scholar 

  5. Roy, A., Livny, M.: Condor and preemptive resume scheduling. In: Nabrzyski, J., Schopf, J.M., Weglarz, J. (eds.) Grid Resource Management: State of the Art and Future Trends, pp. 135–144. Kluwer Academic, Dordrecht (2003)

    Google Scholar 

  6. Natrajan, A., Nguyen-Tuong, A., Humphrey, M.A., Grimshaw, A.S.: The legion grid portal. In: Concurrency and Computation: Practice and Experience, vol. 14, pp. 13–14. Grid Computing Environments Special Issue (2002)

  7. Cooper, K., Dasgupata, A., Kennedy, K., Koelbel, C., Mandal, A., Marin, G., Mazina, M., Mellor-Crummey, J., Berman, F., Casanova, H., Chien, A., Dail, H., Liu, X., Olugbile, A., Sievert, O., Xia, H., Johnsson, L., Liu, B., Patel, M., Reed, D., Deng, W., Mendes, C., Shi, Z., YarKhan, A., Dongarra, J.: New grid scheduling and rescheduling methods in the GrADS project. In: NSF Next Generation Software Workshop, International Parallel and Distributed Processing Symposium, Santa Fe, April 2004

  8. Deelman, E., Singh, G., Su, M.-H., Blythe, J., Gil, Y., Kesselman, C., Mehta, G., Vahi, K., Berriman, G.B., Good, J., Laity, A., Jacob, J.C., Katz, D.S.: Pegasus: a framework for mapping complex scientific workflows onto distributed systems. Sci. Program. J. 13(3), 219–237 (2005)

    Google Scholar 

  9. http://taverna.sourceforge.net/

  10. Young, L., McGough, S., Newhouse, S., Darlington, J.: Scheduling architecture and algorithms within the ICENI grid middleware. Scheduling Architecture and Algorithms within the ICENI Grid Middleware. In All Hands Meeting, UK e-Science Program, Nottingham, 2003

  11. Sulistio, A., Schiffmann, W., Buyya, R.: Advanced reservation-based scheduling of task graphs on clusters. In: Proceedings of the 13th Annual IEEE International Conference on High Performance Computing (HiPC 2006), Dec. 18–21, 2006, Bangalore, India. LNCS, vol. 4297. Springer, Berlin (2006)

    Google Scholar 

  12. Tsafrir, D., Etsion, Y., Feitelson, D.G.: Backfilling using system-generated predictions rather than user runtime estimates. IEEE Trans. Parallel Distributed Syst. 18(6) (2007)

  13. Tsafrir, D., Etsion, Y., Feitelson, D.G.: Modeling user runtime estimates. In: Feitelson, D.G., Frachtenberg, E., Rudolph, L., Schwiegelshohn, U. (eds.) 11th Workshop on Job Scheduling Strategies for Parallel Processing (JSSPP), June 2005. Lecture Notes in Computer Science, vol. 3834, pp. 1–35. Springer, Berlin (2005)

    Chapter  Google Scholar 

  14. Zhang, Y., Mandal, A., Casanova, H., Chien, A.A., Kee, Y., Kennedy, K., Koelbel, C.: Scalable grid application scheduling via decoupled resource selection and scheduling. In: IEEE International Symposium on Cluster Computing and the Grid (CCGrid 2006)

  15. Aziz, A., El-Rewini, H.: Management architecture for resource services and task allocation in grid environments. Technical Report 07-CSE-02, Dept. of Computer Science and Engineering, Southern Methodist University

  16. Braun, T., Siegel, H.J., Beck, N., Boloni, L.L., Maheswaran, M., Reuther, A.I., Robertson, J.P., Theys, M.D., Yao, B., Hengsen, D., Freund, R.F.: A comparison study of static mapping heuristics for a class of meta-tasks on heterogeneous computing systems. In: Proceedings of the International Heterogenous Computing Workshop (HCW’99), pp. 15–29, April, 1999

  17. Singh, H., Youssef, A.: Mapping and scheduling heterogeneous task graphs using genetic algorithms. In: Proceeding of 5th Heterogeneous Computing Workshop (1996)

  18. Shroff, P., Watson, D.W., Flann, N.S., Freund, R.: Genetic simulated annealing for scheduling data-dependent tasks in heterogeneous environments. In: Proceeding of 5th Heterogeneous Computing Workshop (1996)

  19. Roy, A. et al.: A distributed resource management architecture that support advance reservation and co-allocation. In: Proceedings of the International Workshop on Quality of Service (1999)

  20. Michalewicz, Z., Fogel, D.B.: How to Solve It: Modern Heuristics, 2 edn. Springer, Berlin (2004)

    MATH  Google Scholar 

  21. Parallel Workload Archive. Logs of real parallel workloads from production systems, July 2007. http://www.cs.huji.ac.il/labs/parallel/workload/logs.html

  22. Topcuoglu, H., Hariri, S., Wu, M.-Y.: Performance effective and low-complexity task scheduling for heterogeneous computing. IEEE Trans. Parallel Distributed Syst. 13(3), 260–274 (2002)

    Article  Google Scholar 

  23. YarKhan, A., Dongarra, J.J.: Experiments with scheduling using simulated annealing in a grid environment. In: M. Parashar (ed.) Proceedings Grid Computing—GRID 2002, Third International Workshop,Baltimore, MD, USA, November 18, 2002. Lecture Notes in Computer Science, vol. 2536, pp. 232–242. Springer, Berlin (2002)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Southern Methodist University, Dallas, TX, USA

    Abdul Aziz & Hesham El-Rewini

Authors
  1. Abdul Aziz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hesham El-Rewini
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Abdul Aziz.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Aziz, A., El-Rewini, H. On the use of meta-heuristics to increase the efficiency of online grid workflow scheduling algorithms. Cluster Comput 11, 373–390 (2008). https://doi.org/10.1007/s10586-008-0062-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10586-008-0062-y

Keywords

Search

Navigation