Skip to main content

Advertisement

SpringerLink
Log in

Reducing the upfront cost of private clouds with clairvoyant virtual machine placement

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Although public clouds still occupy the largest portion of the total cloud infrastructure, private clouds are attracting increasing interest from both industry and academia because of their better security and privacy control. According to the existing studies, the high upfront cost is among the most critical challenges associated with private clouds. To reduce cost and improve performance, virtual machine placement (VMP) methods have been extensively investigated; however, few of these methods have focused on private clouds. This paper proposes a heterogeneous and multidimensional clairvoyant dynamic bin-packing model, in which the scheduler can conduct more efficient VMP processes using additional information on the arrival time and duration of virtual machines to reduce the datacenter scale and thereby decrease the upfront cost of private clouds. In addition, a novel branch-and-bound algorithm with a divide-and-conquer strategy (DCBB) is proposed to effectively and efficiently handle the derived problem. One state-of-the-art and several classic VMP methods are also modified to adapt to the proposed model to observe their performance and compare with our proposed algorithm. Extensive experiments are conducted on both real-world and synthetic workloads to evaluate the accuracy and efficiency of the algorithms. The experimental results demonstrate that DCBB delivers near-optimal solutions with a convergence rate that is much faster than those of the other search-based algorithms evaluated. In particular, DCBB yields the optimal solution for a real-world workload with an execution time that is an order of magnitude shorter than that required by the original branch-and-bound algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. https://aws.amazon.com/.

  2. https://aws.amazon.com/ec2/.

  3. http://en.inspur.com/inspur/.

  4. https://www.openstack.org/.

References

  1. Mell P, Grance T et al (2011) The NIST definition of cloud computing

  2. Framingham M (2017) Spending on IT infrastructure for public cloud deployments will return to double-digit growth in 2017, according to IDC; 2017. https://www.idc.com/getdoc.jsp?containerId=prUS42454117

  3. Kim W (2017) Cloud computing trends: 2017 state of the cloud survey. https://www.rightscale.com/blog/cloud-industry-insights/cloud-computing-trends-2017-state-cloud-survey. Accessed 23 Jan 2018

  4. Goyal S (2014) Public vs private vs hybrid vs community-cloud computing: a critical review. Int J Comput Netw Inf Secur 6(3):20

    Google Scholar 

  5. Ficco M, Di Martino B, Pietrantuono R, Russo S (2017) Optimized task allocation on private cloud for hybrid simulation of large-scale critical systems. Future Gener Comput Syst 74:104–118

    Article  Google Scholar 

  6. Ramanathan R, Latha B (2018) Towards optimal resource provisioning for hadoop-mapreduce jobs using scale-out strategy and its performance analysis in private cloud environment. Clust Comput. https://doi.org/10.1007/s10586-018-2234-8

  7. Ye X, Li J, Liu S, Liang J, Jin Y (2017) A hybrid instance-intensive workflow scheduling method in private cloud environment. Nat Comput. https://doi.org/10.1007/s11047-016-9600-3

  8. Toosi AN, Vanmechelen K, Ramamohanarao K, Buyya R (2015) Revenue maximization with optimal capacity control in infrastructure as a service cloud markets. IEEE Trans Cloud Comput 3(3):261–274

    Article  Google Scholar 

  9. de Assuncao MD, Lefèvre L (2017) Bare-metal reservation for cloud: an analysis of the trade off between reactivity and energy efficiency. Clust Comput. https://doi.org/10.1007/s10586-017-1094-y

  10. Masdari M, Nabavi SS, Ahmadi V (2016) An overview of virtual machine placement schemes in cloud computing. J Netw Comput Appl 66:106–127

    Article  Google Scholar 

  11. Feldman J, Liu N, Topaloglu H, Ziya S (2014) Appointment scheduling under patient preference and no-show behavior. Oper Res 62(4):794–811

    Article  MathSciNet  MATH  Google Scholar 

  12. Irwin DE, Chase JS, Grit LE, Yumerefendi AR, Becker D, Yocum K (2006) Sharing networked resources with brokered leases. In: USENIX Annual Technical Conference, General Track, pp 199–212

  13. Lawson BG, Smirni E (2002) Multiple-queue backfilling scheduling with priorities and reservations for parallel systems. In: Workshop on Job Scheduling Strategies for Parallel Processing, Springer, pp 72–87

  14. Elmroth E, Tordsson J (2009) A standards-based grid resource brokering service supporting advance reservations, coallocation, and cross-grid interoperability. Concurr Comput Pract Exp 21(18):2298–2335

    Article  Google Scholar 

  15. Farooq U, Majumdar S, Parsons EW (2005) Impact of laxity on scheduling with advance reservations in grids. In: 13th IEEE International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems, 2005. IEEE, pp 319–322

  16. Chase J, Niyato D (2017) Joint optimization of resource provisioning in cloud computing. IEEE Trans Serv Comput 10(3):396–409

    Article  Google Scholar 

  17. Coffman EG Jr, Garey MR, Johnson DS (1983) Dynamic bin packing. SIAM J Comput 12(2):227–258

    Article  MathSciNet  MATH  Google Scholar 

  18. Park JW, Kim E (2017) Runtime prediction of parallel applications with workload-aware clustering. J Supercomput 73(11):4635–4651

    Article  Google Scholar 

  19. Calheiros RN, Masoumi E, Ranjan R, Buyya R (2015) Workload prediction using arima model and its impact on cloud applications’ QoS. IEEE Trans Cloud Comput 3(4):449–458

    Article  Google Scholar 

  20. Gandhi A, Chen Y, Gmach D, Arlitt M, Marwah M (2012) Hybrid resource provisioning for minimizing data center SLA violations and power consumption. Sustain Comput Inf Syst 2(2):91–104

    Google Scholar 

  21. Usmani Z, Singh S (2016) A survey of virtual machine placement techniques in a cloud data center. Procedia Comput Sci 78:491–498

    Article  Google Scholar 

  22. Panigrahy R, Talwar K, Uyeda L, Wieder U (2011) Heuristics for vector bin packing. research microsoft com

  23. Gao Y, Guan H, Qi Z, Hou Y, Liu L (2013) A multi-objective ant colony system algorithm for virtual machine placement in cloud computing. J Comput Syst Sci 79(8):1230–1242

    Article  MathSciNet  Google Scholar 

  24. Tang M, Pan S (2015) A hybrid genetic algorithm for the energy-efficient virtual machine placement problem in data centers. Neural Process Lett 41(2):211–221

    Article  Google Scholar 

  25. Fard SYZ, Ahmadi MR, Adabi S (2017) A dynamic VM consolidation technique for QoS and energy consumption in cloud environment. J Supercomput 73(10):4347–4368

    Article  Google Scholar 

  26. Zheng Q, Li R, Li X, Shah N, Zhang J, Tian F, Chao KM, Li J (2016) Virtual machine consolidated placement based on multi-objective biogeography-based optimization. Fut Gener Comput Syst 54:95–122

    Article  Google Scholar 

  27. Xiao Z, Jiang J, Zhu Y, Ming Z, Zhong S, Cai S (2015) A solution of dynamic VMs placement problem for energy consumption optimization based on evolutionary game theory. J Syst Softw 101:260–272

    Article  Google Scholar 

  28. Vu HT, Hwang S (2014) A traffic and power-aware algorithm for virtual machine placement in cloud data center. Int J Grid Distrib Comput 7(1):350–355

    Article  Google Scholar 

  29. Kanagavelu R, Lee BS, Mingjie LN, Aung KMM et al (2014) Virtual machine placement with two-path traffic routing for reduced congestion in data center networks. Comput Commun 53:1–12

    Article  Google Scholar 

  30. Gupta MK, Amgoth T (2018) Resource-aware virtual machine placement algorithm for IaaS cloud. J Supercomput 74(1):122–140

    Article  Google Scholar 

  31. Liang Q, Zhang J, Zhang Yh, Jm Liang (2014) The placement method of resources and applications based on request prediction in cloud data center. Inf Sci 279:735–745

    Article  Google Scholar 

  32. Sayeedkhan PN, Balaji S (2014) Virtual Machine placement based on disk I/O load in cloud. Int J Comput Sci Inf Technol 5:5477–5479

    Google Scholar 

  33. Xu M, Tian W, Buyya R (2017) A survey on load balancing algorithms for virtual machines placement in cloud computing. Concurr Comput Pract Exp 29(12):e4123

    Article  Google Scholar 

  34. Anand A, Lakshmi J, Nandy S (2013) Virtual machine placement optimization supporting performance SLAs. In: 2013 IEEE 5th International Conference on Cloud Computing Technology and Science (CloudCom), IEEE, vol 1, pp 298–305

  35. Chaisiri S, Lee BS, Niyato D (2009) Optimal virtual machine placement across multiple cloud providers. In: IEEE Asia-Pacific Services Computing Conference, 2009. APSCC 2009. IEEE, pp 103–110

  36. Ribas BC, Suguimoto RM, Montano RA, Silva F, de Bona L, Castilho MA (2012) On modelling virtual machine consolidation to pseudo-Boolean constraints. In: Ibero-American Conference on Artificial Intelligence, Springer, pp 361–370

  37. Fang S, Kanagavelu R, Lee BS, Foh CH, Aung KMM (2013) Power-efficient virtual machine placement and migration in data centers. In: IEEE International Conference on Green Computing and Communications (GreenCom), 2013 IEEE and Internet of Things (iThings/CPSCom) and IEEE Cyber, Physical and Social Computing, IEEE, pp 1408–1413

  38. Dong J, Wang H, Jin X, Li Y, Zhang P, Cheng S (2013) Virtual machine placement for improving energy efficiency and network performance in IaaS cloud. In: 2013 IEEE 33rd International Conference on Distributed Computing Systems Workshops (ICDCSW), IEEE, pp 238–243

  39. Moreno IS, Yang R, Xu J, Wo T (2013) Improved energy-efficiency in cloud datacenters with interference-aware virtual machine placement. In: 2013 IEEE Eleventh International Symposium on Autonomous Decentralized Systems (ISADS), IEEE, pp 1–8

  40. Jp Luo, Li X, Mr Chen (2014) Hybrid shuffled frog leaping algorithm for energy-efficient dynamic consolidation of virtual machines in cloud data centers. Expert Syst Appl 41(13):5804–5816

    Article  Google Scholar 

  41. Liu XF, Zhan ZH, Deng JD, Li Y, Gu T, Zhang J (2016) An energy efficient ant colony system for virtual machine placement in cloud computing. IEEE Trans Evolut Comput

  42. Quang-Hung N, Nien PD, Nam NH, Tuong NH, Thoai N (2013) A genetic algorithm for power-aware virtual machine allocation in private cloud. In: Information and Communication Technology-EurAsia Conference, Springer, pp 183–191

  43. Agrawal K, Tripathi P (2015) Power aware artificial bee colony virtual machine allocation for private cloud systems. In: 2015 International Conference on Computational Intelligence and Communication Networks (CICN), IEEE, pp 947–950

  44. Shi L, Butler B, Botvich D, Jennings B (2013) Provisioning of requests for virtual machine sets with placement constraints in IaaS clouds. In: 2013 IFIP/IEEE International Symposium on Integrated Network Management (IM 2013), IEEE, pp 499–505

  45. Coffman Jr EG, Csirik J, Galambos G, Martello S, Vigo D (2013) Bin packing approximation algorithms: survey and classification. In: Handbook of Combinatorial Optimization, Springer, pp 455–531

  46. De La Vega WF, Lueker GS (1981) Bin packing can be solved within 1+ \(\varepsilon \) in linear time. Combinatorica 1(4):349–355

    Article  MathSciNet  MATH  Google Scholar 

  47. Bansal N, Correa JR, Kenyon C, Sviridenko M (2006) Bin packing in multiple dimensions: inapproximability results and approximation schemes. Math Oper Res 31(1):31–49

    Article  MathSciNet  MATH  Google Scholar 

  48. Han BT, Diehr G, Cook JS (1994) Multiple-type, two-dimensional bin packing problems: applications and algorithms. Ann Oper Res 50(1):239–261

    Article  MathSciNet  MATH  Google Scholar 

  49. Li Y, Tang X, Cai W (2014) On dynamic bin packing for resource allocation in the cloud. In: Proceedings of the 26th ACM Symposium on Parallelism in Algorithms and Architectures, ACM, pp 2–11

  50. Kamali S, López-Ortiz A (2015) Efficient online strategies for renting servers in the cloud. In: International Conference on Current Trends in Theory and Practice of Informatics, Springer, pp 277–288

  51. Tang X, Li Y, Ren R, Cai W (2016) On first fit bin packing for online cloud server allocation. In: 2016 IEEE International Parallel and Distributed Processing Symposium, IEEE, pp 323–332

  52. Ren R, Tang X (2016) Clairvoyant dynamic bin packing for job scheduling with minimum server usage time. In: Proceedings of the 28th ACM Symposium on Parallelism in Algorithms and Architectures, ACM, pp 227–237

  53. Azar Y, Vainstein D (2017) Tight bounds for clairvoyant dynamic bin packing. In: Proceedings of the 29th ACM Symposium on Parallelism in Algorithms and Architectures, ACM, pp 77–86

  54. Gu C, Chen S, Zhang J, Huang H, Jia X (2017) Reservation schemes for IaaS cloud broker: a time-multiplexing way for different rental time. Concurr Comput Pract Exp. https://doi.org/10.1002/cpe.3972

  55. Feitelson D (2017) Parallel workloads archive. http://www.cs.huji.ac.il/labs/parallel/workload

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Technology, Harbin Institute of Technology, Heilongjiang, China

    Yan Zhao, Hongwei Liu, Zhan Zhang & Decheng Zuo

  2. Department of Computing, Macquarie University, Sydney, Australia

    Yan Wang

Authors
  1. Yan Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hongwei Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Decheng Zuo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hongwei Liu.

Additional information

The work described in this paper was supported by the National High-tech R&D Program of China (863 Program) under Grant 2013AA01A215 and the National Laboratory of High-effect Server and Storage Techniques under Grant 2014HSSA05.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhao, Y., Liu, H., Wang, Y. et al. Reducing the upfront cost of private clouds with clairvoyant virtual machine placement. J Supercomput 75, 340–369 (2019). https://doi.org/10.1007/s11227-018-02730-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-018-02730-4

Keywords

Search

Navigation