Advertisement

Cluster Computing

, Volume 5, Issue 3, pp 237–246 | Cite as

Condor-G: A Computation Management Agent for Multi-Institutional Grids

  • James Frey
  • Todd Tannenbaum
  • Miron Livny
  • Ian Foster
  • Steven Tuecke
Article

Abstract

In recent years, there has been a dramatic increase in the number of available computing and storage resources. Yet few tools exist that allow these resources to be exploited effectively in an aggregated form. We present the Condor-G system, which leverages software from Globus and Condor to enable users to harness multi-domain resources as if they all belong to one personal domain. We describe the structure of Condor-G and how it handles job management, resource selection, security, and fault tolerance. We also present results from application experiments with the Condor-G system. We assert that Condor-G can serve as a general-purpose interface to Grid resources, for use by both end users and higher-level program development tools.

Condor Globus distributed computing Grid computing 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [1]
    D. Abramson, J. Giddy and L. Kotler, High performance parametric modeling with Nimrod/G: Killer application for the Global Grid~, in: IPDPS'2000 (IEEE Press, 2000).Google Scholar
  2. [2]
    D. Abramson, R. Sosic, J. Giddy and B. Hall, Nimrod: A tool for performing parameterized simulations using distributed workstations, in: Proc. 4th IEEE Symp. on High Performance Distributed Computing (1995).Google Scholar
  3. [3]
    C. Anglano et al. Integrating GRID tools to build a computing resource broker: Activities of DataGrid WP1, in: Computing in High Energy and Nuclear Physic (2001).Google Scholar
  4. [4]
    K. Anstreicher, N. Brixius, J.-P. Goux and J. Linderoth, Solving large quadratic assignment problems on computational Grids, in: Mathematical Programming (2000). CONDOR-G 245Google Scholar
  5. [5]
    K. Anstreicher, N. Brixius, J.-P. Goux and J. Linderoth, Solving large quadratic assignment problems on computational Grids, in: Mathematical Programming (to appear).Google Scholar
  6. [6]
    J. Beiriger, W. Johnson, H. Bivens, S. Humphreys and R. Rhea, Constructing the ASCI Grid, in: Proc. 9th IEEE Symposium on High Performance Distributed Computing (IEEE Press, 2000).Google Scholar
  7. [7]
    F. Berman, High-performance schedulers, in: The Grid: Blueprint for a New Computing Infrastructure, eds. I. Foster and C. Kesselman (Morgan Kaufmann, 1999) pp. 279-309.Google Scholar
  8. [8]
    F. Berman, R.Wolski, S. Figueira, J. Schopf and G. Shao, Applicationlevel scheduling on distributed heterogeneous networks, in: Proc. Supercomputing '96 (1996).Google Scholar
  9. [9]
    J. Bester, I. Foster, C. Kesselman, J. Tedesco and S. Tuecke, GASS: A data movement and access service for wide area computing systems, in: 6th Workshop on I/O in Parallel and Distributed Systems, 5 May 1999.Google Scholar
  10. [10]
    H. Casanova, G. Obertelli, F. Berman and R. Wolski, The AppLeS parameter sweep template: User-level middleware for the Grid, in: Proc. SC'2000 (2000).Google Scholar
  11. [11]
    K. Czajkowski, S. Fitzgerald, I. Foster and C. Kesselman, Grid information services for distributed resource sharing, in: Proc. 10th IEEE Int. Symp. on High-Performance Distributed Computing (2001) pp. 181-184.Google Scholar
  12. [12]
    K. Czajkowski, I. Foster, N. Karonis, C. Kesselman, S. Martin, W. Smith and S. Tuecke, A resource management architecture for metacomputing systems, in: Proc. IPPS/SPDP '98 Workshop on Job Scheduling Strategies for Parallel Processing (1998).Google Scholar
  13. [13]
    D.H.J. Epema, M. Livny, R.V. Dantzig, X. Evers and J. Pruyne, A worldwide flock of condors: Load sharing among workstation clusters, Future Generation Computer Systems 12 (1996).Google Scholar
  14. [14]
    I. Foster and C. Kesselman, Globus: A toolkit-based Grid architecture, in: The Grid: Blueprint for a New Computing Infrastructure, eds. I. Foster and C. Kesselman (Morgan Kaufmann, 1999) pp. 259-278.Google Scholar
  15. [15]
    I. Foster, C. Kesselman, G. Tsudik and S. Tuecke, A security architecture for computational Grids, in: ACM Conference on Computers and Security (1998) pp. 83-91.Google Scholar
  16. [16]
    I. Foster, C. Kesselman and S. Tuecke, The anatomy of the Grid: Enabling scalable virtual organizations, Int. J. High Performance Computing Applications 15(3) (2001) 200–222, http://www.globus.org/ research/papers/anatomy.pdf.Google Scholar
  17. [17]
    J. Gray and A. Reuter, Two-phase commit: Making computations atomic, in: Transaction Processing: Concepts and Techniques (Morgan Kaufmann, 1993) pp. 562-573.Google Scholar
  18. [18]
    R. Henderson and D. Tweten, Portable Batch System: External Reference Specification (1996).Google Scholar
  19. [19]
    IBM, Using and Administering IBM LoadLeveler, Release 3.0, IBM CorporationSC23-3989 (1996).Google Scholar
  20. [20]
    Institute S.C.R., DQS 3.1.3 User Guide, Florida State University, Tallahassee (1996).Google Scholar
  21. [21]
    W.E. Johnston, D. Gannon and B. Nitzberg, Grids as production computing environments: The engineering aspects of NASA's information power Grid, in: Proc. 8th IEEE Symposium on High Performance Distributed Computing (IEEE Press, 1999).Google Scholar
  22. [22]
    E. Korpela, D. Werthimer, D. Anderson, J. Cobb and M. Lebofsky, SETI@home: Massivel distributed computing for SETI, Computing in Science and Engineering 3(1) (2001).Google Scholar
  23. [23]
    G. von Laszewski, I. Foster, J. Gawor and J. Lane, A Java commodity Grid toolkit, Concurrency: Practice and Experience 13 (2001) (to appear).Google Scholar
  24. [24]
    M. Litzkow, M. Livny and M. Mutka, Condor - A hunter of idle workstations, in: Proc. 8th Int. Conf. on Distributed Computing Systems (1988) pp. 104-111.Google Scholar
  25. [25]
    M. Litzkow, T. Tannenbaum, J. Basney and M. Livny, Checkpoint and migration of UNIX processes in the Condor distributed processing system, University of Wisconsin-Madison Computer Sciences, Technical Report 1346 (1997).Google Scholar
  26. [26]
    M. Livny, High-throughput resource management, in: The Grid: Blueprint for a New Computing Infrastructure, eds. I. Foster and C. Kesselman (Morgan Kaufmann, 1999) pp. 311-337.Google Scholar
  27. [27]
    NCSA Alliance, Grid-in-a-Box, http://www.ncsa.uiuc.edu/TechFocus/ Deployment/GiB.Google Scholar
  28. [28]
    NCSA Alliance, GridGaussian, http://www.ncsa.uiuc.edu/Divisions/ ACES/APG/grid_gaussian.htm.Google Scholar
  29. [29]
    J. Novotny, S. Tuecke and V. Welch, An online credential repository for the Grid: MyProxy, in: Proc. 10th IEEE Int. Symp. on High-Performance Distributed Computing (2001).Google Scholar
  30. [30]
    M. Papakhian, Comparing job-management systems: The user's perspective, IEEE Computational Science & Engineering (April-June) (1998), http://pbs.mrj.com.Google Scholar
  31. [31]
    R. Raman, M. Livny and M. Solomon, Resource management through multilateral matchmaking, in: Proc. of the 9th IEEE Symposium on High Performance Distributed Computing (HPDC9), Pittsburgh, Pennsylvania (August 2000) pp. 290-291.Google Scholar
  32. [32]
    J. Steiner, B.C. Neuman and J. Schiller, Kerberos: An authentication system for open network systems, in: Proc. Usenix Conference (1988) pp. 191-202.Google Scholar
  33. [33]
    R. Stevens, P. Woodward, T. DeFanti and C. Catlett, From the I-WAY to the national technology Grid, Communications of the ACM 40(11) (1997) 50–61.Google Scholar
  34. [34]
    S. Vazhkudai, S. Tuecke and I. Foster, Replica selection in the Globus data Grid, in: Proc. of the 1st IEEE/ACM Int. Conference on Cluster Computing and the Grid (CCGRID 2001) (IEEE Computer Society Press, 2001) pp. 106-113.Google Scholar
  35. [35]
    S. Zhou, LSF: Load sharing in large-scale heterogeneous distributed systems, in: Proc. Workshop on Cluster Computing (1992).Google Scholar
  36. [36]
    Condor Project, Globus ASCII Helper Protocol (GAHP), http://www.cs.wisc.edu/condor/gaph.Google Scholar

Copyright information

© Kluwer Academic Publishers 2002

Authors and Affiliations

  • James Frey
    • 1
  • Todd Tannenbaum
    • 1
  • Miron Livny
    • 1
  • Ian Foster
    • 2
  • Steven Tuecke
    • 2
  1. 1.Department of Computer ScienceUniversity of WisconsinMadisonUSA
  2. 2.Mathematics and Computer Science DivisionArgonne National LaboratoryArgonneUSA

Personalised recommendations