New Grid Scheduling and Rescheduling Methods in the GrADS Project

  • F. Berman
  • H. Casanova
  • A Chien
  • K. Cooper
  • H. Dail
  • A. Dasgupta
  • W. Deng
  • J. Dongarra
  • L. Johnsson
  • K. Kennedy
  • C. Koelbel
  • B. Liu
  • X. Liu
  • A. Mandal
  • G. Marin
  • M. Mazina
  • J. Mellor-Crummey
  • C. Mendes
  • A. Olugbile
  • M. Patel
  • D. Reed
  • Z. Shi
  • O. Sievert
  • H. Xia
  • A. YarKhan
Article

Abstract

The goal of the Grid Application Development Software (GrADS) Project is to provide programming tools and an execution environment to ease program development for the Grid. This paper presents recent extensions to the GrADS software framework: a new approach to scheduling workflow computations, applied to a 3-D image reconstruction application; a simple stop/migrate/restart approach to rescheduling Grid applications, applied to a QR factorization benchmark; and a process-swapping approach to rescheduling, applied to an N-body simulation. Experiments validating these methods were carried out on both the GrADS MacroGrid (a small but functional Grid) and the MicroGrid (a controlled emulation of the Grid).

Keywords

Grid computing scheduling rescheduling 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    I. Foster and C. Kesselman (eds.), The Grid: Blueprint for a New Computing Infrastructure, 2nd Ed., Morgan Kaufmann (2003)Google Scholar
  2. 2.
    K. Kennedy, M. Mazina, J. Mellor-Crummey, K. Cooper, L Torczon, F. Berman, A. Chien, H. Dail, O. Sievert, D. Angulo, I Foster, D. Gannon, S. L. Johnsson, C. Kesselman, R. Aydt, D. Reed, J. Dongarra, S. Vadhiyar, and R. Wolski, Towards a Framework for Preparing and Executing Adaptive Grid Programs, Proceedings of NSF Next Generation Systems Program Workshop (International Parallel and Distributed Processing Symposium), Fort Lauderdale, Florida (April 2002)Google Scholar
  3. 3.
    Ribler, R.L., Simitci, H., Reed, D.A. September 2001The Autopilot Performance-directed Adaptive Control SystemFuture Generation Computer Systems.1817518kk7Google Scholar
  4. 4.
    Vraalsen F., Aydt R.A., Mendes C.L., Reed D.A. Performance Contracts: Predicting and Monitoring Grid Application Behavior, Lecture Notes in Computer Science, Vol. 2242, pp. 154–165, Springer Verlag (November 2001)Google Scholar
  5. 5.
    H. Song, X. Liu, D. Jakobsen, R. Bhagwan, X. Zhang, K. Taura, and A. Chien, The MicroGrid: A Scientific Tool for Modeling Computational Grids, Proceedings of SC2000 (November 2000)Google Scholar
  6. 6.
    O. Sievert and H. Casanova, Policies for Swapping MPI Processes, Proceedings of HPDC-12, the Symposium on High Performance and Distributed Computing (June 2003)Google Scholar
  7. 7.
    Barish B., Weiss R. (1999). Ligo and detection of gravitational waves. Physics Today. 52(10)Google Scholar
  8. 8.
    S. Hastings, T. Kurc, S. Langella, U. Catalyurek, T. Pan, and J. Saltz, Image Processing on the Grid: A Toolkit or Building Grid-enabled Image Processing Applications, 3rd International Symposium on Cluster Computing and the Grid (2003)Google Scholar
  9. 9.
    K. Taura and A. Chien, A Heuristic Algorithm for Mapping Communicating Tasks on Heterogeneous Resources, Heterogeneous Computing Workshop (May 2000)Google Scholar
  10. 10.
    S. Vadhiyar and J. Dongarra, A Metascheduler for the Grid, Proceedings of the High Performance Distributed Computing Conference (July 2002)Google Scholar
  11. 11.
    R. Wolski, J. Plank, J. Brevik, and T. Bryan, G-commerce: Market Formulations Controlling Resource Allocation on the Computational Grid, Proceedings of 2001 International Parallel and Distributed Processing Symposium (1PDPS) (March 2001) Google Scholar
  12. 12.
    Condor Team, Condor Version 6.4.7 Manual, //www.cs.wisc.edu/condor/ manual/v6.4/Google Scholar
  13. 13.
    S. Fitzgerald, I. Foster, C. Kesselman, G. von Laszewski, W Smith, and S. Tuecke, A Directory Service for Configuring High-Performance Distributed Computations, Proceedings of the 6th IEEE Symposium on High-Performance Distributed Computing, pp 365–375 (August 1997), URL papers/fitzgerald–hpdc97-mds.pdfGoogle Scholar
  14. 14.
    Wolski, R., Spring, N.T., Hayes, J. 1999The network weather service: a distributed resource performance forecasting service for metacomputingFuture Generation Computer Systems.15757768URL citeseer.nj.nec.com/wolski98network.htmlGoogle Scholar
  15. 15.
    M.R. Garey and D.S. Johnson, Computers and Intractability: A Guide to the Theory of Np-Completeness, MIT Press (1979)Google Scholar
  16. 16.
    H. Casanova, A. Legrand, D. Zagorodnov, and F. Berman, Heuristics for Scheduling Parameter Sweep applications in Grid environments, 9th Heterogeneous Computing workshop (HCW’2000) (2000)Google Scholar
  17. 17.
    Tracy, D., Braun, ,  et al. 2001A Comparision of eleven Static Heuristics for Maping a Class of Independent Tasks onto Heterogeneous Distributed Computing SystemsJournal of Parallel and Distributed Computing.61810837Google Scholar
  18. 18.
    G. Marin, Semi-Automatic Synthesis of Parameterized Performance Models for Scientific Programs, Master’s thesis, Department of Computer Science, Rice University (April 2003)Google Scholar
  19. 19.
    Ludtke, S., Baldwin, P., Chiu, W. 1999EMAN: Semiautomated Software for High- Resolution Single-Particle ReconstructionsJ. Struct. Biol.1288297URL http: //ncmi.bcm.tmc.edu/homes/stevel/EMAN/docGoogle Scholar
  20. 20.
    Vadhiyar, S., Dongarra, J. June 2003SRS A Framework for Developing Malleable and Migratable Parallel Applications for Distributed SystemsParallel Processing Letters.13291312iSSN 0129-6264Google Scholar
  21. 21.
    J.S. Plank, M. Beck, W. Elwasif, T. Moore, M. Swany, and R Wolski, The Internet Backplane Protocol: Storage in the Network, NetStore99: The Network Storage Symposium (1999)Google Scholar
  22. 22.
    L.S. Blackford, J. Choi, A. Cleary, E. D’Azevedo, J. Demmel, I. Dhillon, J. Dongarra, S. Hammerling, G. Henry, A. Petitet, K Stanley, D. Walker, and R.C. Whaley, ScaLAPACK User’s Guide. (1997)Google Scholar
  23. 23.
    S. Vadhiyar and J. Dongarra, A Performance Oriented Migration Framework for the Grid, IEEE Computing Clusters and the Grid (CCGrid, http://www.ccgrid.org) (May 12–15 2003)Google Scholar
  24. 24.
    O. Sievert and H. Casanova, A Simple MPI Process Swapping Architecture for Iterative Applications, The International Journal of High Performance Computing Applications (2004), to appearGoogle Scholar
  25. 25.
    X. Liu and A. Chien, Traffic-based Load Balance for Scalable Network Emulation, Proceedings of SC2003 (November 2003)Google Scholar
  26. 26.
    H. Xia, H. Dail, H. Casanova, F. Berman, and A. Chien, Evaluating the GrADS Scheduler in Diverse Grid Environments Using the MicroGrid (May 2003), submitted for publicationGoogle Scholar
  27. 27.
    A. Petitet, S. Blackford, J. Dongarra, B. Ellis, G. Fagg, K Roche, and S. Vadhiyar, Numerical Libraries and the Grid, Proceedings of SC’01 (November 2001)Google Scholar
  28. 28.
    M. Ripeanu, A. Iamnitchi, and I. Foster, Cactus Application: Performance Predictions in a Grid Environment, Proceedings of European Conference on Parallel Computing (EuroPar)2001 (August 2001)Google Scholar
  29. 29.
    W. Chrabakh and R. Wolski, GrADSAT: A Parallel SAT Solver for the Grid, Technical Report CS-2003-05, University of California, Santa Barbara (2003), available from http://www.cs.ucsb.edu/research/trcs/index.shtmlGoogle Scholar
  30. 30.
    H. Dail, A Modular Framework for Adaptive Scheduling in Grid Application Development Environments, Master’s thesis, University of California, San Diego, Department of Computer Science and Engineering (Mardh 2002), available as UCSD Technical Report CS2002-0698Google Scholar
  31. 31.
    A. Mandal, Mapping HPF onto the Grid, Technical report TR03-417, Department of Computer Science, Rice University, Houston (November 2002), URL http://www.cs. rice edu/anirban/MSthesis.ps.gzGoogle Scholar

Copyright information

© Springer Science+Business Media, Inc. 2005

Authors and Affiliations

  • F. Berman
    • 1
  • H. Casanova
    • 1
  • A Chien
    • 1
  • K. Cooper
    • 2
  • H. Dail
    • 1
  • A. Dasgupta
    • 2
  • W. Deng
    • 3
  • J. Dongarra
    • 4
  • L. Johnsson
    • 5
  • K. Kennedy
    • 2
  • C. Koelbel
    • 2
  • B. Liu
    • 5
  • X. Liu
    • 1
  • A. Mandal
    • 2
  • G. Marin
    • 2
  • M. Mazina
    • 2
  • J. Mellor-Crummey
    • 2
  • C. Mendes
    • 3
  • A. Olugbile
    • 1
  • M. Patel
    • 5
  • D. Reed
    • 6
  • Z. Shi
    • 4
  • O. Sievert
    • 1
  • H. Xia
    • 1
  • A. YarKhan
    • 4
  1. 1.Department of Computer Science and EngineeringUniversity of California at San DiegoSan DiegoUSA
  2. 2.Computer Science Dept.Rice UniversityHoustonUSA
  3. 3.Department Computer ScienceUniversity of IllinoisUrbanaUSA
  4. 4.Innovative Computing LabUniversity of TennesseeKnoxvilleUSA
  5. 5.Department Computer ScienceUniversity of HoustonHoustonUSA
  6. 6.Renaissance Computing InstituteUniversity of North Carolina at Chapel HillChapel HillUSA

Personalised recommendations