Resource stealing: a resource multiplexing method for mix workloads in cloud system

Abstract

The cloud computing paradigm enables providing resources on demand. However, most of them focus on a single type of application requiring separate quality of service. In the context that mix heterogeneous workloads are co-scheduled in the cloud, resource multiplexing is the key to improve resource utilization under premise of performance guaranteing. In this paper, we propose a resource stealing mechanism to improve resource multiplexing of cloud resources. It enables free resource fragments reserved by some workloads being utilized by others. To meet certain service level agreement, resource preemption is adopted as a complement to resource stealing. It ensures each workload with a minimum amount of resources when required. Moreover, we propose an adaptive joint resource provisioning algorithm. It integrates our resource multiplexing method into elastic resource provisioning. Experimental results reveal that the proposed algorithms improve resource utilization and workload performance simultaneously.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

References

  1. 1.

    Buyya R, Yeo CS, Venugopal S, Broberg J, Brandic I (2009) Cloud computing and emerging IT platforms: vision, hype, and reality for delivering computing as the 5th utility. Future Gener Comput Syst 25(6):599–616

    Article  Google Scholar 

  2. 2.

    Staples G (2006) TORQUE resource manager. In: Proceedings of the 2006 ACM/IEEE conference on supercomputing. ACM, p 8

  3. 3.

    Yang XJ, Liao XK, Lu K (2011) The TianHe-1A supercomputer: its hardware and software[J]. J Comput Sci Technol V26(3):344–351

    Article  Google Scholar 

  4. 4.

    Liao XK, Xiao LQ, Yang CQ et al (2014) MilkyWay-2 supercomputer: system and application[J]. Front Comput Sci 8(3):345–356

    MathSciNet  Article  Google Scholar 

  5. 5.

    Vavilapalli VK, Murthy AC, Douglas C, Agarwal S, Konar M, Evans R, Baldeschwieler E (2013) Apache hadoop yarn: yet another resource negotiator. In: Proceedings of the 4th annual symposium on cloud computing. ACM, p 5

  6. 6.

    Hindman B, Konwinski A, Zaharia M, Ghodsi A, Joseph AD, Katz R, Stoica I (2011) Mesos: a platform for fine-grained resource sharing in the data center. In: Proceedings of the 8th USENIX conference on networked systems design and implementation, p 22

  7. 7.

    Ghribi C, Zeghlache D (2014) Exact and heuristic graph-coloring for energy efficient advance cloud resource reservation. In: Cloud computing (CLOUD), 2014 IEEE 7th international conference on, June 27–July 2 2014, pp 112–119

  8. 8.

    Isard M, Budiu M, Yu Y, Birrell A, Fetterly D (2007) Dryad: distributed data-parallel programs from sequential building blocks. ACM SIGOPS Oper Syst Rev 41(3):59–72

    Article  Google Scholar 

  9. 9.

    Xu X, Dou W, Zhang X, Chen J (2015) EnReal: an energy-aware resource allocation method for scientific workflow executions in cloud environment. Cloud Comput IEEE Trans (99):1. doi:10.1109/TCC.2015.2453966

  10. 10.

    Dean J, Ghemawat S (2008) MapReduce: simplified data processing on large clusters. Commun ACM 51(1):107–113

    Article  Google Scholar 

  11. 11.

    Wang WT, Xu L, Gupta I (2015) Scale Up vs. scale out in cloud storage and graph processing systems. In: Cloud engineering (IC2E), 2015 IEEE international conference on, 9–13 March 2015, pp 428-433. doi:10.1109/IC2E.2015.55

  12. 12.

    Ekanayake J, Li H, Zhang B, Gunarathne T, Bae SH, Qiu J, Fox G (2010) Twister: a runtime for iterative mapreduce. In: Proceedings of the 19th ACM international symposium on high performance distributed computing. ACM, pp 810–818

  13. 13.

    Murray DG, Schwarzkopf M, Smowton C, Smith S, Madhavapeddy A, Hand S (2011) CIEL: a universal execution engine for distributed data-flow computing. NSDI

  14. 14.

    Yuan XY, Tang HY, Li Y, Jia T, Liu TC, Wu ZH (2015) A competitive penalty model for availability based cloud SLA. In: Cloud computing (CLOUD), 2015 IEEE 8th International Conference on, June 27–July 2 2015, pp 964–970

  15. 15.

    Liang Z, Sakr S, Liu A (2015) A Framework for consumer-centric sla management of cloud-hosted databases. In: Services computing, IEEE Transactions on, July–Aug 2015, vol 8, no 4, pp 534–549. doi:10.1109/TSC.2013.5

  16. 16.

    Tootaghaj DZ, Farhat F, Arjomand M, Faraboschi P, Kandemir MT, Sivasubramaniam A, Das CR (2015) Evaluating the combined impact of node architecture and cloud workload characteristics on network traffic and performance/cost. In: Workload characterization (IISWC), 2015 IEEE international symposium on, 4–6 Oct 2015, pp 203–212. doi:10.1109/IISWC.2015.31

  17. 17.

    Caglar F, Gokhale A (2014) iOverbook: intelligent resource-overbooking to support soft real-time applications in the cloud. In: Cloud computing (CLOUD), 2014 IEEE 7th international conference on, June 27–July 2 2014, pp 538–545. doi:10.1109/CLOUD.2014.78

  18. 18.

    Ranger C, Raghuraman R, Penmetsa A, Bradski G, Kozyrakis C (2007) Evaluating mapreduce for multi-core and multiprocessor systems. In: High performance computer architecture, 2007. HPCA 2007. IEEE 13th international symposium on. IEEE, pp 13–24

  19. 19.

    Guo Z, Fox G, Zhou M, Ruan Y (2012) Improving resource utilization in mapreduce. In: Cluster computing (CLUSTER), 2012 IEEE international conference on. IEEE, pp 402–410

  20. 20.

    Garg SK, Gopalaiyengar SK, Buyya R (2011) SLA-based resource provisioning for heterogeneous workloads in a virtualized cloud datacenter. In: Algorithms and architectures for parallel processing. Springer, Berlin, Heidelberg, pp 371–384

  21. 21.

    Meng X, Isci C, Kephart J, Zhang L, Bouillet E, Pendarakis D (2010) Efficient resource provisioning in compute clouds via vm multiplexing. In: Proceedings of the 7th international conference on autonomic computing. ACM, pp 11–20

  22. 22.

    Calheiros RN, Ranjan R, Buyya R (2011) Virtual machine provisioning based on analytical performance and QoS in cloud computing environments. In: Proceedings of the 2011 international conference on parallel processing (ICPP), pp 295–304, 13–16 Sept 2011

Download references

Acknowledgments

This work is supported by project (2013AA01A212) from the National 863 Program of China, project (61202121) from the National Natural Science Foundation of China, Science and technology project (2013Y2-00043) in Guangzhou of China.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Fuhui Wu.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Tan, Y., Wu, F., Wu, Q. et al. Resource stealing: a resource multiplexing method for mix workloads in cloud system. J Supercomput 75, 33–49 (2019). https://doi.org/10.1007/s11227-015-1609-3

Download citation

Keywords

  • Resource multiplexing
  • Resource stealing
  • Resource preemption
  • Performance guaranteing
  • Mix workloads