Skip to main content
SpringerLink
Log in

Dimensioning and on-line scheduling in Lambda Grids using divisible load concepts

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Due to the large amounts of data required to be processed by the typical Grid job, it is conceivable that the use of optical transport networks in Grid deployment (hence the term “Lambda Grid”) will increase. The exact topology of the interconnecting network is obtained by solving a dimensioning problem, and the outcome of this strongly depends on both the expected workload characteristics and Grid scheduling policy. Solving this combined scheduling and dimensioning problem using straightforward ILP modelling is cumbersome; however, for steady-state Grid operation, Divisible Load Theory (DLT) can yield scalable formulations of this problem.

In this paper, the on-line hierarchical scheduling on a lambda Grid of workload approaching the Grid’s capacity in a two-tier Grid mode of operation is studied. A number of these algorithms are goal-driven, in the sense that target per-resource goals are obtained from the off-line solution to the Divisible Load model. We compare these on-line multiresource scheduling policies for different workloads, Grid interconnection topologies and Grid parameters. We show that these algorithms perform well in the studied scenarios when compared to a fully centralized scheduling algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Foster I, Kesselman C (eds) (1999) The grid: blueprint for a new computing infrastructure. Morgan Kaufmann

  2. Foster I, Kesselman C (eds) (2003) The grid: blueprint for a new computing infrastructure, 2nd edn. Morgan Kaufmann

  3. Smarr L, Ford J, Papadopoulos P, Fainman S, DeFanti T, Brown M, Leigh J (2005) The optiputer, quartzite, and starlight projects: a campus to global-scale testbed for optical technologies enabling lambdagrid computing. In: Optical fiber communication conference & exposition and the national fiber optic engineers conference (OFC/NFOEC) 2005, 2005

  4. DeFanti T, de Laat C, Mambretti J, Neggers K, Arnaud BS (2003) Translight: a global-scale lambdagrid for e-science. Commun ACM 46(11):34–41

    Article  Google Scholar 

  5. Hall L, Schulz A, Shmoys D, Wein J (1997) Mathematics of Operations Research. Scheduling to minimize average completion time: off-line and on-line approximation algorithms. Math Oper Res 22(3)

  6. Feitelson D, Rudolph L, Schwiegelshohn U, Sevcik K, Wong P (1997) Theory and practice in parallel job scheduling. Springer

  7. Sgall J (1998) On-line scheduling—a survey. Lecture notes in computer science, vol 1442

  8. Kolish R, Padman R (2001) An integrated survey of project scheduling. OMEGA Int J Manag Sci 29(3)

  9. Ranganathan K, Foster I (2003) Simulation studies of computation and data scheduling algorithms for data grids. J Grid Comput 1(1)

  10. Cameron D, Carvajal-Schiaffino R, Millar A, Nicholson C, Stockinger K, Zini F (2003) Evaluating scheduling and replica optimisation strategies in optorsim. In: 4th International workshop on grid computing (Grid2003), 2003

  11. Ernemann C, Hamscher V, Streit A, Yahyapour R (2002) Enhanced algorithms for multi-site scheduling. In: 3rd International workshop on grid computing (Grid2002), 2002

  12. Bucur A, Epema D (2002) An evaluation of processor co-allocation for different system configurations and job structures. In: 14th IEEE symposium on computer architecture and high performance computing, 2002

  13. Bucur A, Epema D (2000) The influence of the structure and sizes of jobs on the performance of co-allocation. In: 6th Workshop on job scheduling strategies for parallel processing, 2000

  14. Buyya R, Murshed M, Abramson D (2002) A deadline and budget constrained cost-time optimization algorithm for scheduling task farming applications on global grids. In: The 2002 International conference on parallel and distributed processing techniques and applications (PDPTA’02), 2002

  15. Thysebaert P, Volckaert B, Turck FD, Dhoedt B, Demeester P (2004) Network aspects of grid scheduling algorithms. In: 17th International conference on parallel and distributed computing systems (PDCS’04), 2004

  16. Hung J, Kim H, Robertazzi T (2002) Scalable scheduling in parallel processors. In: 36th Annual conference on information sciences and systems (CISS’02), 2002

  17. Yu D, Robertazzi, T (2003) Divisible load scheduling for grid computing. In: 16th International conference on parallel and distributed computing systems (PDCS’03), 2003

  18. Marchal L, Yang Y, Casanova H, Robert Y (2005) A realistic network/application model for scheduling divisible loads on large-scale platforms. In: 19th IEEE international parallel and distributed processing symposium (IPDPS’05), 2005

  19. Keren A, Barak A (2003) Opportunity cost algorithms for reduction of i/o and interprocess communication overhead in a computing cluster. IEEE Trans Parallel Distrib Syst 14(1):39–50

    Article  Google Scholar 

  20. Farooq U, Majumdar S, Parsons E (2005) Dynamic scheduling of lightpaths in lambda grids. In: IEEE/create-net international workshop on networks for grid applications (GRIDNETS 2005), 2005

  21. Zhu Y (2003) A survey on grid scheduling systems, Ph.D. Qualifying exam. Hong Kong University of Science and Technology

  22. Thysebaert P, Leenheer MD, Volckaert B, Turck FD, Dhoedt B, Demeester P (2006) Scalable dimensioning of optical transport networks for grid excess load handling. Photon Netw Commun 12(2):117–132

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Information Technology, Ghent University—IBBT—IMEC, Gaston Crommenlaan 8, 9050, Gent, Belgium

    Pieter Thysebaert, Bruno Volckaert, Marc De Leenheer, Filip De Turck, Bart Dhoedt & Piet Demeester

Authors
  1. Pieter Thysebaert
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bruno Volckaert
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Marc De Leenheer
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Filip De Turck
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Bart Dhoedt
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Piet Demeester
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pieter Thysebaert.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Thysebaert, P., Volckaert, B., De Leenheer, M. et al. Dimensioning and on-line scheduling in Lambda Grids using divisible load concepts. J Supercomput 42, 59–82 (2007). https://doi.org/10.1007/s11227-006-0036-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-006-0036-x

Keywords

Search

Navigation