Advertisement

The Journal of Supercomputing

, Volume 42, Issue 1, pp 59–82 | Cite as

Dimensioning and on-line scheduling in Lambda Grids using divisible load concepts

  • Pieter ThysebaertEmail author
  • Bruno Volckaert
  • Marc De Leenheer
  • Filip De Turck
  • Bart Dhoedt
  • Piet Demeester
Article

Abstract

Due to the large amounts of data required to be processed by the typical Grid job, it is conceivable that the use of optical transport networks in Grid deployment (hence the term “Lambda Grid”) will increase. The exact topology of the interconnecting network is obtained by solving a dimensioning problem, and the outcome of this strongly depends on both the expected workload characteristics and Grid scheduling policy. Solving this combined scheduling and dimensioning problem using straightforward ILP modelling is cumbersome; however, for steady-state Grid operation, Divisible Load Theory (DLT) can yield scalable formulations of this problem.

In this paper, the on-line hierarchical scheduling on a lambda Grid of workload approaching the Grid’s capacity in a two-tier Grid mode of operation is studied. A number of these algorithms are goal-driven, in the sense that target per-resource goals are obtained from the off-line solution to the Divisible Load model. We compare these on-line multiresource scheduling policies for different workloads, Grid interconnection topologies and Grid parameters. We show that these algorithms perform well in the studied scenarios when compared to a fully centralized scheduling algorithm.

Keywords

Lambda Grids Optical Transport Networks Optimization Divisible Load 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Foster I, Kesselman C (eds) (1999) The grid: blueprint for a new computing infrastructure. Morgan Kaufmann Google Scholar
  2. 2.
    Foster I, Kesselman C (eds) (2003) The grid: blueprint for a new computing infrastructure, 2nd edn. Morgan Kaufmann Google Scholar
  3. 3.
    Smarr L, Ford J, Papadopoulos P, Fainman S, DeFanti T, Brown M, Leigh J (2005) The optiputer, quartzite, and starlight projects: a campus to global-scale testbed for optical technologies enabling lambdagrid computing. In: Optical fiber communication conference & exposition and the national fiber optic engineers conference (OFC/NFOEC) 2005, 2005 Google Scholar
  4. 4.
    DeFanti T, de Laat C, Mambretti J, Neggers K, Arnaud BS (2003) Translight: a global-scale lambdagrid for e-science. Commun ACM 46(11):34–41 CrossRefGoogle Scholar
  5. 5.
    Hall L, Schulz A, Shmoys D, Wein J (1997) Mathematics of Operations Research. Scheduling to minimize average completion time: off-line and on-line approximation algorithms. Math Oper Res 22(3) Google Scholar
  6. 6.
    Feitelson D, Rudolph L, Schwiegelshohn U, Sevcik K, Wong P (1997) Theory and practice in parallel job scheduling. Springer Google Scholar
  7. 7.
    Sgall J (1998) On-line scheduling—a survey. Lecture notes in computer science, vol 1442 Google Scholar
  8. 8.
    Kolish R, Padman R (2001) An integrated survey of project scheduling. OMEGA Int J Manag Sci 29(3) Google Scholar
  9. 9.
    Ranganathan K, Foster I (2003) Simulation studies of computation and data scheduling algorithms for data grids. J Grid Comput 1(1) Google Scholar
  10. 10.
    Cameron D, Carvajal-Schiaffino R, Millar A, Nicholson C, Stockinger K, Zini F (2003) Evaluating scheduling and replica optimisation strategies in optorsim. In: 4th International workshop on grid computing (Grid2003), 2003 Google Scholar
  11. 11.
    Ernemann C, Hamscher V, Streit A, Yahyapour R (2002) Enhanced algorithms for multi-site scheduling. In: 3rd International workshop on grid computing (Grid2002), 2002 Google Scholar
  12. 12.
    Bucur A, Epema D (2002) An evaluation of processor co-allocation for different system configurations and job structures. In: 14th IEEE symposium on computer architecture and high performance computing, 2002 Google Scholar
  13. 13.
    Bucur A, Epema D (2000) The influence of the structure and sizes of jobs on the performance of co-allocation. In: 6th Workshop on job scheduling strategies for parallel processing, 2000 Google Scholar
  14. 14.
    Buyya R, Murshed M, Abramson D (2002) A deadline and budget constrained cost-time optimization algorithm for scheduling task farming applications on global grids. In: The 2002 International conference on parallel and distributed processing techniques and applications (PDPTA’02), 2002 Google Scholar
  15. 15.
    Thysebaert P, Volckaert B, Turck FD, Dhoedt B, Demeester P (2004) Network aspects of grid scheduling algorithms. In: 17th International conference on parallel and distributed computing systems (PDCS’04), 2004 Google Scholar
  16. 16.
    Hung J, Kim H, Robertazzi T (2002) Scalable scheduling in parallel processors. In: 36th Annual conference on information sciences and systems (CISS’02), 2002 Google Scholar
  17. 17.
    Yu D, Robertazzi, T (2003) Divisible load scheduling for grid computing. In: 16th International conference on parallel and distributed computing systems (PDCS’03), 2003 Google Scholar
  18. 18.
    Marchal L, Yang Y, Casanova H, Robert Y (2005) A realistic network/application model for scheduling divisible loads on large-scale platforms. In: 19th IEEE international parallel and distributed processing symposium (IPDPS’05), 2005 Google Scholar
  19. 19.
    Keren A, Barak A (2003) Opportunity cost algorithms for reduction of i/o and interprocess communication overhead in a computing cluster. IEEE Trans Parallel Distrib Syst 14(1):39–50 CrossRefGoogle Scholar
  20. 20.
    Farooq U, Majumdar S, Parsons E (2005) Dynamic scheduling of lightpaths in lambda grids. In: IEEE/create-net international workshop on networks for grid applications (GRIDNETS 2005), 2005 Google Scholar
  21. 21.
    Zhu Y (2003) A survey on grid scheduling systems, Ph.D. Qualifying exam. Hong Kong University of Science and Technology Google Scholar
  22. 22.
    Thysebaert P, Leenheer MD, Volckaert B, Turck FD, Dhoedt B, Demeester P (2006) Scalable dimensioning of optical transport networks for grid excess load handling. Photon Netw Commun 12(2):117–132 CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2007

Authors and Affiliations

  • Pieter Thysebaert
    • 1
    Email author
  • Bruno Volckaert
    • 1
  • Marc De Leenheer
    • 1
  • Filip De Turck
    • 1
  • Bart Dhoedt
    • 1
  • Piet Demeester
    • 1
  1. 1.Department of Information TechnologyGhent University—IBBT—IMECGentBelgium

Personalised recommendations