The Journal of Supercomputing

, Volume 68, Issue 2, pp 509–556 | Cite as

Semantic-enabled CARE Resource Broker (SeCRB) for managing grid and cloud environment

  • Thamarai Selvi Somasundaram
  • Kannan Govindarajan
  • Usha Kiruthika
  • Rajkumar Buyya
Article

Abstract

Grid computing is mainly helpful for executing high-performance computing applications. However, conventional grid resources sometimes fail to offer a dynamic application execution environment and this increases the rate at which the job requests of users are rejected. Integrating emerging virtualization technologies in grid and cloud computing facilitates the provision of dynamic virtual resources in the required execution environment. Resource brokers play a significant role in managing grid and cloud resources as well as identifying potential resources that satisfy users’ application requests. This research paper proposes a semantic-enabled CARE Resource Broker (SeCRB) that provides a common framework to describe grid and cloud resources, and to discover them in an intelligent manner by considering software, hardware and quality of service (QoS) requirements. The proposed semantic resource discovery mechanism classifies the resources into three categories viz., exact, high-similarity subsume and high-similarity plug-in regions. To achieve the necessary user QoS requirements, we have included a service level agreement (SLA) negotiation mechanism that pairs users’ QoS requirements with matching resources to guarantee the execution of applications, and to achieve the desired QoS of users. Finally, we have implemented the QoS-based resource scheduling mechanism that selects the resources from the SLA negotiation accepted list in an optimal manner. The proposed work is simulated and evaluated by submitting real-world bio-informatics and image processing application for various test cases. The result of the experiment shows that for jobs submitted to the resource broker, job rejection rate is reduced while job success and scheduling rates are increased, thus making the resource management system more efficient.

Keywords

Cloud computing Grid computing High-performance computing (HPC) Resource broker Semantic description Semantic discovery Service level agreement (SLA) 

List of symbols

NS

Number of Schedule

\(S\,\leftarrow \,\left\{ {S_1 ,S_2 , \ldots ,S_p.S_\mathrm{NS}} \right\} \)

It represents set of schedules

\(S_j \)

It represents pth schedule

M

It represents the total number of resources

\(R\leftarrow \{{R_1 ,R_2,\ldots ,R_M} \}\)

It represents the list of resources

\(J_i\)

It represents ith job

\(R_j\)

It represents jth resource

CB

Cloud broker

\((\mathrm{ER})_{J_i}\)

Exact region contain resources that exactly satisfies the demands of job \(J_i\)

\((\mathrm{SR})_{J_i}\)

Subsume region contain resources that has more capability than demand of job \(J_i\)

\((\mathrm{PR})_{J_i}\)

Plug-in region contain resources that has less capability than demand of Job \(J_i\)

\(\mathrm {Sim\_mat}\)

Similarity matrix holds the cosine similar measure between jobs and resources in \((\mathrm{SR})_{J_i}\) and \((\mathrm{PR})_{J_i}\). Its dimension are \(({1 *\left| {(\mathrm{SR})_{J_i}} \right| })\) and \(({1*\left| {(\mathrm{PR})_{J_i}} \right| })\), respectively

\(({\mathrm{hs}\_\mathrm{SR}})_{J_i}\)

High-similarity resources that satisfy demand specified by job \(J_i\) from subsume region

\(({\mathrm{hs}\_\mathrm{PR}})_{J_i}\)

High-similarity resources that satisfy demand specified by job \(J_i \) from plug-in region

\(\mathrm{threshold}_\mathrm{SR}\)

To filter the resources from subsume region whose similarity is greater or equal to specified value

\(\mathrm{threshold}_\mathrm{PR}\)

To filter the resources from plug-in region whose similarity is greater or equal to specified value

\(\mathrm{Req}({J_i})\)

\(\{J_i^\mathrm{ID} ,\,\mathrm{no}\_\mathrm{Nodes}_{J_i} ,\,\mathrm{Ram}\_\mathrm{needed}_{J_i} ,\mathrm{Hd}\_\mathrm{needed}_{J_i }, \mathrm{Deadline}_{J_i} ,\,\mathrm{OS}_{J_i} ,\,\mathrm{Sw}_{J_i} \}\)

\(\mathrm{Avail}({R_j})\)

\(\left\{ \!\!{\begin{array}{l} R_j^\mathrm{ID} ,\mathrm{no}\_\mathrm{Nodes}_{R_j}, \mathrm{Ram}\_\mathrm{Capacity}_{R_j}, \mathrm{Hd}\_\mathrm{Capacity}_{R_j},\\ \mathrm{Availability}_{R_j},\mathrm{OS}_{R_j} ,\mathrm{Sw}_{R_j}\!, \mathrm{Type}_{R_j} \\ \end{array}} \!\!\right\} \)

\(\left| {(\mathrm{ER})_{J_i}} \right| \)

Number of resources in exact region of job \(J_i \) that is represented as \(N\_\mathrm{ER}\)

\(\left| {\,({\mathrm{hs}\_\mathrm{SR}})_{J_i} \,} \right| \)

Number of resources in subsume region of job \(J_i \) that is represented as \(N\_\mathrm{SR}\)

\(\left| {\,({\mathrm{hs}\_\mathrm{PR}})_{J_i} \,} \right| \)

Number of resources in plug-in region that satisfies job \(J_i \) that is represented as \(N\_\mathrm{PR}\)

\(N_{(\mathrm{ER})_{J_i}}\)

List of resources to which negotiation is agreed in exact region by broker

\(N_{(\mathrm{SR})_{J_i}}\)

List of resources to which negotiation is agreed in subsume region by broker

\(N_{(\mathrm{PR})_{J_i}}\)

List of resources to which negotiation is agreed in plug-in region by broker

\(P_\mathrm{Init}\)

Initial proposal send by cloud broker

\(U_\mathrm{Init}\)

Utility value of the cloud broker

\(({P_\mathrm{c}})_{({R_j})}\)

Counter proposal given by resource \(R_j\) that receives \(P_\mathrm{Init}\)

\((\mathrm{No. proposal})_\mathrm{ER}\)

Number of resources in exact region that send counter proposals which is received by broker

\((\mathrm{No. proposal})_\mathrm{SR}\)

Number of resources in subsume region that send counter proposals which is received by broker

\((\mathrm{No. proposal})_\mathrm{PR}\)

Number of resources in plug-in region that send counter proposals which is received by broker

\(({U_\mathrm{c}})_{({R_j})}\)

Utility counter of the resource \(R_j\)

\(U_{(\mathrm{ER})_{J_i}}\)

Set of utilities received by broker from the resources in exact region of the job \(J_i \)

\(U_{(\mathrm{SR})_{J_i}}\)

Set of utilities received by broker from the resources in qualified subsume region of the job \(J_i \)

\(U_{(\mathrm{PR})_{J_i}}\)

Set of utilities received by broker from the resources in qualified plug-in region of the job \(J_i \)

\(P_\mathrm{Avg}\)

Average proposal received by the cloud broker

\(({R_{(\mathrm{PR})}})_{J_i}\)

Resources in plug-in region that is scheduled to Job \(J_i \)

\(\mathrm{JS}_{S_p}\)

Number of Jobs submitted for the Schedule \(S_p \)

\(\mathrm{JR}_{S_p}\)

Number of Jobs rejected for the schedule\(S_p \)

\(\mathrm{JRR}_{S_p}\)

Job rejection Rate for the schedule \(S_p \)

\(\mathrm{JSTS}_{S_p}\)

Number of Jobs submitted to the scheduler in the schedule \(S_p\)

\(\mathrm{JSR}_{S_p}\)

Job Success Rate for the schedule \(S_p \)

\(\mathrm{JSD}_{S_p}\)

Number of Jobs successfully met deadline in the schedule \(S_p \)

\(\mathrm{JFD}_{S_p}\)

Number of jobs failed to meet deadline in the schedule\(S_p \)

\(\mathrm{SSR}_{S_p}\)

Scheduling success rate for the schedule \(S_p \)

\(\hbox {TPDefected}_{\mathrm{ith}\_\mathrm{fruit}}\)

Total pixels defected in ith fruit

References

  1. 1.
    Price Water House Coopers (2002) Powerful technology trends continue despite downturn. PWC Global Technology Center, Menlo ParkGoogle Scholar
  2. 2.
    National Institute of Standards and Technology (NIST). http://csrc.nist.gov/publications/drafts/800-145/Draft-SP-800-145_Cloud-definition.pdf. Accessed Jan 2013
  3. 3.
  4. 4.
    Multitenancy. http://theCloudtutorial.com/multitenancy.html. Accessed Jan 2013
  5. 5.
    Czajkowski K, Foster I, Karonis N, Kesselman C, Martin S, Smith W, Tuecke S (1998) A resource management architecture for metacomputing systems. In: Proceedings of the workshop on job scheduling strategies for parallel processing. Springer, Berlin, Heidelberg, pp 62–82Google Scholar
  6. 6.
    Missier P, Ziegler W, Wieder P (2007) Semantic support for meta-scheduling in grids. In: Talia D, Bilas A, Dikaiakos MD (eds) Knowledge and Data Management in GRIDs. Springer, USA, pp 169–183Google Scholar
  7. 7.
    Berners-Lee T, Hendler J, Lassila O (2001) The semantic web. Sci Am 284(5):34–43Google Scholar
  8. 8.
  9. 9.
    Goble C, De Roure D (2004) The Semantic Grid: Myth Busting and Bridge Building. In: Proceedings of the 16th European Conference on Artificial Intelligence (ECAI 2004). Valencia, SpainGoogle Scholar
  10. 10.
    Resource Description Framework (RDF) (2004) A W3C recommendation. http://www.w3.org/RDF
  11. 11.
    Web Ontology Language (OWL). http://www.w3.org/2004/OWL. Accessed Jan 2013
  12. 12.
    Protégé (2011) http://protege.stanford.edu/. Accessed Jan 2013
  13. 13.
    Somasundaram TS, Balachander RA, Kumar R, Balakrishnan P, Rajendar K, Rajiv R, Kannan G, Rajesh Britto GR, Mahendran E, Madusudhanan B (2010) CARE resource broker: a framework for scheduling and supporting virtual resource management. Futur Gener Comput Syst 26(3): 337–347. doi:10.1016/j.future.2009.10.005
  14. 14.
    Globus Toolkit. http://www.globus.org/toolkit/downloads/4.0.7/. Accessed Oct 2012
  15. 15.
    Somasundaram TS, Balachander RA, Balakrishnan P, Kumar R, Rajendar K, Rajiv R, Rajesh Britto G, Eahendran E, Madusudhanan B (2009) Achieving co-allocation through virtualization in grid environment. In: Abdennadher N, Petcu D (eds) Advances in grid and pervasive computing, vol 5529. Springer, Berlin, Heidelberg, pp 235–243Google Scholar
  16. 16.
    Somasundaram TS, Balachandar RA (2009) Extending conventional grid scheduler for leasing resources. In 10th IEEE/ACM International Conference in Grid Computing 2009 (Grid 2009), Banff, Canada, pp: 173–174Google Scholar
  17. 17.
    Amarnath BR, Somasundaram TS, Mahendran E, Buyya R (2009) Ontology based grid resource management. Softw Pract Exper, vol 39 (17): pp 1419–1438Google Scholar
  18. 18.
    Somasundaram TS, Rangasamy K, Govindarajan K (2011) Intelligent semantic discovery in virtualized grid environment. In: International Conference on Recent Trends in Information Technology (ICRTIT), 3–5 June 2011, Chennai, Tamil Nadu pp 644–649Google Scholar
  19. 19.
    The Condor Project. http://research.cs.wisc.edu/condor/. Accessed Nov 2012
  20. 20.
    Computing Resource Execution and Management (CREAM) (2012) http://grid.pd.infn.it/cream/
  21. 21.
    gLite Workload Management System (WMS) (2012). http://iopscience.iop.org/1742-6596/219/6/062039/pdf/1742-6596_219_6_062039.pdf. Accessed Nov 2012
  22. 22.
    Classified Advertisements (ClassAds). http://research.cs.wisc.edu/condor/classad/. Accessed Nov 2012
  23. 23.
  24. 24.
    Grid Interoperability Project. http://www.Grid-interoperability.eu/. Accessed Jan 2013
  25. 25.
    InteliGrid (2004). http://inteliGrid.eu-project.info/. Accessed Jan 2013
  26. 26.
    The myGrid Project (2008). http://www.myGrid.org.uk. Accessed Jan 2013
  27. 27.
    Roure D, Jennings N, Shadbolt N (2003) The semantic grid: a future e-science infrastructure. In: Berman F, Fox G, Hey AJG (eds) Grid Computing - Making the Global Infrastructure a Reality. Wiley, New YorkGoogle Scholar
  28. 28.
    Somasundaram TS, Balachander RA, Kandasamy V, Buyya R, Rajagopalan Raman M, Varun S (2006) Semantic-based grid resource discovery and its integration with the grid service broker. In: Proceedings of the 14th International Conference on Advanced Computing and Communications (ADCOM), pp 84–89Google Scholar
  29. 29.
    Harth A, Decker S, He Y, Tangmunarunkit H, Kesselman C (2004) A semantic matchmaker service on the grid. In: Proceedings of the 13th international World Wide Web conference on Alternate track papers & posters, May 17–20, 2004, New York, pp 326–327Google Scholar
  30. 30.
    Nurmi D, Wolski R, Grzegorczyk C, Obertelli G, Soman S, Youseff L, Zagorodnov D (2009) The Eucalyptus open-source cloud-computing system. In: IEEE/ACM International Symposium on Cluster Computing and the Grid (CCGRID’09), Shanghai, pp 124–131Google Scholar
  31. 31.
    OpenNebula. http://www.opennebula.org. Accessed Mar 2013
  32. 32.
    Emeneker W, Jackson D, Butikofer J, Stanzione D (2006) Dynamic virtual clustering with Xen and Moab. In: Proceedings of the 2006 international conference on frontiers of high performance computing and networking. Springer, Berlin, Heidelberg, pp 440–451Google Scholar
  33. 33.
    Agarwal A, Desmarais R, Gable I, Norton A, Sobie R, Vanderster D (2006) Evaluation of virtual machines for HEP grids. In: Proceedings of computing in high energy physics, Mumbai, IndiaGoogle Scholar
  34. 34.
    Xen. http://www.xen.org/. Accessed Mar 2013
  35. 35.
    Vmware. http://www.vmware.com/. Accessed Mar 2013
  36. 36.
    Kernel-based Virtual Machine (KVM). http://www.linux-kvm.org/. Accessed Mar 2013
  37. 37.
    Amazon EC2 Cloud. http://www.amazon.org/ec2. Accessed Mar 2013
  38. 38.
    Ma YB, Jang SH, Lee JS (2011) Ontology-based resource management for cloud computing. In: Intelligent information and database systems. Lecture Notes in computer science, vol 6592. Springer, Berlin, pp 343–352Google Scholar
  39. 39.
    Xu B, Wang N, Li C (2010) Providing a cloud infrastructure on clusters. In: Proceedings of the third international symposium on electronic commerce and security workshops (ISECS ’10), Guangzhou, P.R. China, 29–31 July 2010, pp 317–320Google Scholar
  40. 40.
    Buyya R, Pandey S, Vecchiola C (2009) Cloudbus toolkit for market-oriented cloud computing. In: Cloud Computing. Lecture Notes in Computer Science. Springer, BerlinGoogle Scholar
  41. 41.
    Ejarque J, Sirvent R, Badia RM (2010) A multi-agent approach for semantic resource allocation. In: IEEE second international conference on cloud computing technology and science, Indianapolis, pp 335–342Google Scholar
  42. 42.
    Somasundaram TS, Balakrishnan P (2011) SLA Enabled Care Resource Broker. Futur Gener Comput Syst. 27(3):265–279Google Scholar
  43. 43.
    Ejarque J, Palol M, Goiri I, Julia F, Guitart J, Badia RM, Torres J (2009) SLA-driven semantically-enhanced dynamic resource allocator for virtualized service providers. In: IEEE Fourth international conference on eScience, IndianapolisGoogle Scholar
  44. 44.
    Czajkowski K, Foster IT, Kesselman C, Sander V, Tuecke S (2002) SNAP: a protocol for negotiating service level agreements and coordinating resource management in distributed systems. In: 8th International workshop on job scheduling strategies for parallel processing, Edinburgh, Scotland, pp 153–183, July 2002Google Scholar
  45. 45.
    Venugopal S, Chu X, Buyya R (2008) A negotiation mechanism for advance resource reservation using the alternate offers protocol. In: 16th international workshop on quality of service (IWQoS (2008) June 2–4, 2008, Enschede, pp 40–49Google Scholar
  46. 46.
    Yan J, Kowalczyk R, Lin J, Chhetri MB, Goh SK, Zhang J (2007) Autonomous service level agreement negotiation for service composition provision. Futur Gener Comput Syst 23(6):748–759Google Scholar
  47. 47.
    Son S, Sim KM (2011) A negotiation mechanism that facilitates the price-timeslot-QoS negotiation for establishing SLAs of cloud service reservation. In: Fong S (ed) Networked digital technologies. Communications in computer and information science, vol 136. Springer, Berlin, pp 432–446Google Scholar
  48. 48.
    StratusLab. http://stratuslab.eu/doku.php/start. Accessed Feb 2013
  49. 49.
    WNoDeS (2012). http://web.infn.it/wnodes/index.php/wnodes. Accessed Feb 2013
  50. 50.
    GLUE Schema V1.3. http://glueschema.forge.cnaf.infn.it/spec/v13/. Accessed Oct 2012
  51. 51.
    Open Cloud Computing Interface (OCCI). http://occi-wg.org/. Accessed Jan 2013
  52. 52.
    de Assunção MD, di Costanzo A, Buyya R (2010) A cost-benefit analysis of using cloud computing to extend the capacity of clusters. Cluster Comput 13(3): 335–347Google Scholar
  53. 53.
    Ostermann S, Prodan R, Fahringer T (2009) Extending grids with cloud resource management for scientific computing. In: 10th IEEE/ACM International Conference on Grid Computing, Banff, pp 42–49Google Scholar
  54. 54.
    Iosup A, Ostermann S, Yigitbasi N, Prodan R, Fahringer T, Epema DHJ (2011) Performance analysis of cloud computing services for many-tasks scientific computing. IEEE Trans Parallel Distrib Syst 22(6):931–945Google Scholar
  55. 55.
    Mao M, Humphrey M (2012) A performance study on the VM startup time in the cloud, In: 2012 IEEE 5th International Conference on Cloud Computing(CLOUD), Honolulu, pp 423–430Google Scholar
  56. 56.
  57. 57.
    Fast Fourier Transfer (FFT) (2011). http://www.fftw.org/. Accessed Dec 2012
  58. 58.
    Message Passing Interface (MPI). http://www.mcs.anl.gov/research/projects/mpi/. Accessed Dec 2012
  59. 59.
    GROMACS. http://www.gromacs.org/. Accessed Feb 2013
  60. 60.
    Globus Toolkit Auto Install (GTAI) Utility. http://dev.globus.org/wiki/Incubator/GT_Auto_Install. Accessed Dec 2012
  61. 61.
    Protein Data Bank (PDB). http://www.pdb.org/pdb/. Accessed Feb 2013
  62. 62.
    Feitelson D (1997) Parallel workloads archive. http://www.cs.huji.ac.il/labs/parallel/workload. Accessed Jan 2013
  63. 63.
    Irwin DE, Grit LE, Chase JS (2004) Balancing risk and reward in a market-based task service. In: Proceedings of the 13th IEEE International Symposium on High Performance Distributed Computing. Honolulu, USA, pp 160–169Google Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  • Thamarai Selvi Somasundaram
    • 1
  • Kannan Govindarajan
    • 1
  • Usha Kiruthika
    • 1
  • Rajkumar Buyya
    • 2
  1. 1.Anna UniversityChennaiIndia
  2. 2.Cloud Computing and Distributed Systems LabThe University of MelbourneParkvilleAustralia

Personalised recommendations