Journal of Grid Computing

, Volume 8, Issue 1, pp 1–18 | Cite as

Autonomic Clouds on the Grid

  • Michael A. Murphy
  • Linton Abraham
  • Michael Fenn
  • Sebastien Goasguen


Computational clouds constructed on top of existing Grid infrastructure have the capability to provide different entities with customized execution environments and private scheduling overlays. By designing these clouds to be autonomically self-provisioned and adaptable to changing user demands, user-transparent resource flexibility can be achieved without substantially affecting average job sojourn time. In addition, the overlay environment and physical Grid sites represent disjoint administrative and policy domains, permitting cloud systems to be deployed non-disruptively on an existing production Grid. Private overlay clouds administered by, and dedicated to the exclusive use of, individual Virtual Organizations are termed Virtual Organization Clusters. A prototype autonomic cloud adaptation mechanism for Virtual Organization Clusters demonstrates the feasibility of overlay scheduling in dynamically changing environments. Commodity Grid resources are autonomically leased in response to changing private scheduler loads, resulting in the creation of virtual private compute nodes. These nodes join a decentralized private overlay network system called IPOP (IP Over P2P), enabling the scheduling and execution of end user jobs in the private environment. Negligible overhead results from the addition of the overlay, although the use of virtualization technologies at the compute nodes adds modest service time overhead (under 10%) to computationally-bound Grid jobs. By leasing additional Grid resources, a substantial decrease (over 90%) in average job queuing time occurs, offsetting the service time overhead.


Cloud computing Grid computing Autonomic leasing Virtualization Dynamic computational environment 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Murphy, M.A., Fenn, M., Goasguen, S.: Virtual organization clusters. In: 17th Euromicro International Conference on Parallel, Distributed, and Network-Based Processing (PDP 2009), Weimar, Germany (2009)Google Scholar
  2. 2.
    Foster, I., Kesselman, C., Tuecke, S.: The anatomy of the Grid: enabling scalable virtual organizations. Int. J. Supercomput. Appl. 15(3), 200–222 (2001)CrossRefGoogle Scholar
  3. 3.
    Murphy, M.A., Kagey, B., Fenn, M., Goasguen, S.: Dynamic provisioning of virtual organization clusters. In: 9th IEEE International Symposium on Cluster Computing and the Grid (CCGrid ’09), Shanghai, China (2009)Google Scholar
  4. 4.
    Beaumont, O., Carter, L., Ferrante, J., Legrand, A., Marchal, L., Robert, Y.: Centralized versus distributed schedulers for bag-of-tasks applications. IEEE Trans. Parallel Distrib. Syst. 19(5), 698–709 (2008)CrossRefGoogle Scholar
  5. 5.
    Barham, P., Dragovic, B., Fraser, K., Hand, S., Harris, T., Ho, A., Neugebauer, R., Pratt, I., Warfield, A.: Xen and the art of virtualization. In: Nineteenth ACM Symposium on Operating Systems Principles (2003)Google Scholar
  6. 6.
    Fenn, M., Murphy, M.A., Goasguen, S.: A study of a KVM-based cluster for Grid computing. In: 47th ACM Southeast Conference (ACMSE ’09), Clemson, SC (2009)Google Scholar
  7. 7.
    Tannenbaum, T., Wright, D., Miller, K., Livny, M.: Condor—a distributed job scheduler. In: Sterling, T. (ed.) Beowulf Cluster Computing with Linux. MIT, Cambridge (2001)Google Scholar
  8. 8.
    Xu, D., Ruth, P., Rhee, J., Kennell, R., Goasguen, S.: Autonomic adaptation of virtual distributed environments in a multi-domain infrastructure. In: 15th IEEE International Symposium on High Performance Distributed Computing (HPDC’06), Paris, France (2006)Google Scholar
  9. 9.
    Ganguly, A., Agrawal, A., Boykin, P.O., Figueiredo, R.: IP over P2P: enabling self-configuring virtual IP networks for Grid computing. In: 20th International Parallel and Distributed Processing Symposium (IPDPS 2006) (2006)Google Scholar
  10. 10.
    Figueiredo, R., Dinda, P.A., Fortes, J.: Resource virtualization renaissance. Computer 38(5), 28–31 (2005)CrossRefGoogle Scholar
  11. 11.
    Figueiredo, R.J., Dinda, P.A., Fortes, J.A.B.: A case for Grid computing on virtual machines. In: 23rd International Conference on Distributed Computing Systems (2003)Google Scholar
  12. 12.
    Tsugawa, M., Fortes, J.A.B.: A virtual network (ViNe) architecture for Grid computing. In: 20th International Parallel and Distributed Processing Symposium (IPDPS 2006) (2006)Google Scholar
  13. 13.
    Sundararaj, A.I., Dinda, P.A.: Towards virtual networks for virtual machine Grid computing. In: Third Virtual Machine Research and Technology Symposium, San Jose, CA (2004)Google Scholar
  14. 14.
    Davoli, R.: VDE: virtual distributed ethernet. In: First International Conference on Testbeds and Research Infrastructures for the Development of Networks and Communities (Tridentcom 2005), Trento, Italy (2005)Google Scholar
  15. 15.
    Adabala, S., Chadha, V., Chawla, P., Figueiredo, R., Fortes, J., Krsul, I., Matsunaga, A., Tsugawa, M., Zhang, J., Zhao, M., Zhu, L., Zhu, X.: From virtualized resources to virtual computing Grids: the In-VIGO system. Future Gener. Comput. Syst. 21(6), 896–909 (2005)CrossRefGoogle Scholar
  16. 16.
    Keahey, K., Foster, I., Freeman, T., Zhang, X., Galron, D.: Virtual workspaces in the Grid. In: 11th International Euro-Par Conference, Lisbon, Portugal (2005)Google Scholar
  17. 17.
    Sotomayor, B., Keahey, K., Foster, I.: Combining batch execution and leasing using virtual machines. In: 17th International Symposium on High Performance Distributed Computing (HPDC 2008) (2008)Google Scholar
  18. 18.
    Foster, I., Freeman, T., Keahey, K., Scheftner, D., Sotomayor, B., Zhang, X.: Virtual clusters for Grid communities. In: 6th IEEE International Symposium on Cluster Computing and the Grid (CCGrid 2006), Singapore (2006)Google Scholar
  19. 19.
    Irwin, D., Chase, J., Grit, L., Yumerefendi, A., Becker, D., Yocum, K.: Sharing network resources with brokered leases. In: USENIX Technical Conference, Boston, MA (2006)Google Scholar
  20. 20.
    Chase, J.S., Irwin, D.E., Grit, L.E., Moore, J.D., Sprenkle, S.E.: Dynamic virtual clusters in a Grid site manager. In: HPDC ’03: Proceedings of the 12th IEEE International Symposium on High Performance Distributed Computing (2003)Google Scholar
  21. 21.
    Grit, L., Irwin, D., Yumerefendi, A., Chase, J.: Virtual machine hosting for networked clusters: building the foundations for ‘autonomic’ orchestration. In: First International Workshop on Virtualization Technology in Distributed Computing (VTDC ’06), Tampa, FL (2006)Google Scholar
  22. 22.
    Ramakrishnan, L., Grit, L., Iamnitchi, A., Irwin, D., Yumerefendi, A., Chase, J.: Toward a doctrine of containment: Grid hosting with adaptive resource control. In: 19th Annual Supercomputing Conference (SC ’06), Tampa, FL (2006)Google Scholar
  23. 23.
    Ruth, P., McGachey, P., Xu, D.: VioCluster: virtualization for dynamic computational domains. In: IEEE International Conference on Cluster Computing, Boston, MA (2005)Google Scholar
  24. 24.
    Ruth, P., Jiang, X., Xu, D., Goasguen, S.: Virtual distributed environments in a shared infrastructure. Computer 38(5), 63–69 (2005)CrossRefGoogle Scholar
  25. 25.
    Emeneker, W., Stanzione, D.: Dynamic virtual clustering. In: 2007 IEEE International Conference on Cluster Computing (2007)Google Scholar
  26. 26.
    Carns, P.H., Ligon, W.B., Ross, R.B., Thakur, R.: PVFS: a parallel file system for Linux clusters. In: ALS’00: Proceedings of the 4th annual Linux Showcase and Conference (2000)Google Scholar
  27. 27.
    Keahey, K., Freeman, T.: Contextualization: providing one-click virtual clusters. In: 4th IEEE International Conference on e-Science, Indianapolis, IN (2008)Google Scholar
  28. 28.
    Boykin, P.O., Bridgewater, J.S.A., Kong, J.S., Lozev, K.M., Rezaei, B.A., Roychowdhury, V.P.: A symphony conducted by Brunet. Online (2007)Google Scholar
  29. 29.
    Keahey, K., Doering, K., Foster, I.: From sandbox to playground: dynamic virtual environments in the Grid. In: 5th International Workshop on Grid Computing (Grid 2004), Pittsburgh, PA (2004)Google Scholar
  30. 30.
    Pordes, R., Petravick, D., Kramer, B., Olson, D., Livny, M., Roy, A., Avery, P., Blackburn, K., Wenaus, T., Würthwein, F., Foster, I., Gardner, R., Wilde, M., Blatecky, A., McGee, J., Quick, R.: The Open Science Grid: Status and architecture. In: International Conference on Computing in High Energy and Nuclear Physics (CHEP ’07) (2007)Google Scholar
  31. 31.
    Red Hat: Kernel-based Virtual Machine.
  32. 32.
    Foster, I., Kesselman, C.: Globus: a metacomputing infrastructure toolkit. Int. J. Supercomput. Appl. 11(2), 115–128 (1997)CrossRefGoogle Scholar
  33. 33.
    Clemson Cyberinfrastructure Research Group: SimVOC.
  34. 34.
    Sulistio, A., Yeo, C.S., Buyya, R.: A taxonomy of computer-based simulations and its mapping to parallel and distributed systems simulation tools. Int. J. Soft.: Practice and Experience 34(7), 653–673 (2004)CrossRefGoogle Scholar
  35. 35.
    EGEE: Grid observatory.
  36. 36.
    Enabling Grids for E-sciencE: gLite.
  37. 37.
    Sfiligoi, I., Quinn, G., Green, C., Thain, G.: Pilot job accounting and auditing in Open Science Grid. In: 9th IEEE/ACM International Conference on Grid Computing (Grid ’08) (2008)Google Scholar

Copyright information

© Springer Science+Business Media B.V. 2009

Authors and Affiliations

  • Michael A. Murphy
    • 1
  • Linton Abraham
    • 1
  • Michael Fenn
    • 1
  • Sebastien Goasguen
    • 1
  1. 1.School of ComputingClemson UniversityClemsonUSA

Personalised recommendations