The Journal of Supercomputing

, Volume 61, Issue 1, pp 46–66 | Cite as

Power-efficient distributed scheduling of virtual machines using workload-aware consolidation techniques

  • Mohsen SharifiEmail author
  • Hadi Salimi
  • Mahsa Najafzadeh


There is growing demand on datacenters to serve more clients with reasonable response times, demanding more hardware resources, and higher energy consumption. Energy-aware datacenters have thus been amongst the forerunners to deploy virtualization technology to multiplex their physical machines (PMs) to as many virtual machines (VMs) as possible in order to utilize their hardware resources more effectively and save power. The achievement of this objective strongly depends on how smart VMs are consolidated. In this paper, we show that blind consolidation of VMs not only does not reduce the power consumption of datacenters but it can lead to energy wastage. We present four models, namely the target system model, the application model, the energy model, and the migration model, to identify the performance interferences between processor and disk utilizations and the costs of migrating VMs. We also present a consolidation fitness metric to evaluate the merit of consolidating a number of known VMs on a PM based on the processing and storage workloads of VMs. We then propose an energy-aware scheduling algorithm using a set of objective functions in terms of this consolidation fitness metric and presented power and migration models. The proposed scheduling algorithm assigns a set of VMs to a set of PMs in a way to minimize the total power consumption of PMs in the whole datacenter. Empirical results show nearly 24.9% power savings and nearly 1.2% performance degradation when the proposed scheduling algorithm is used compared to when other scheduling algorithms are used.


Power management Virtualization technology Workload characterization Cloud computing 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Koomey J (2007) Estimating total power consumption by servers in the us and the world. Lawrence Berkeley National Laboratory, Stanford University, Berkeley Google Scholar
  2. 2.
    Beloglazov A, Buyya R (2010) Energy efficient resource management in virtualized cloud data centers. In: IEEE, pp 826–831. doi: 10.1109/CCGRID.2010.46 Google Scholar
  3. 3.
    Garg S, Yeo C, Anandasivam A, Buyya R (2010) Environment-conscious scheduling of hpc applications on distributed cloud-oriented data centers. J Parallel Distrib Comput. doi: 10.1016/j.jpdc.2010.04.004
  4. 4.
    Lee YC, Zomaya AY (2010) Energy efficient utilization of resources in cloud computing systems. J Supercomput. doi: 10.1007/s11227-010-0421-3
  5. 5.
    Kim K, Buyya R, Kim J (2007) Power aware scheduling of bag-of-tasks applications with deadline constraints on dvs-enabled clusters. In: Citeseer, pp 541–548 Google Scholar
  6. 6.
    Co A (2008) Data centre energy forecast report, final report. Accenture Co, Silicon Valley Leadership Group Google Scholar
  7. 7.
    Malone C, Belady C (2006) Metrics to characterize data center & it equipment energy use. In: Digital power forum, Richardson, TX Google Scholar
  8. 8.
    Smith J, Nair R (2005) Virtual machines: versatile platforms for systems and processes. Morgan Kaufmann, San Mateo zbMATHGoogle Scholar
  9. 9.
    Foster I, Zhao Y, Raicu I, Lu S (2009) Cloud computing and grid computing 360-degree compared. In: IEEE, pp 1–10. doi: 10.1109/GCE.2008.4738445 Google Scholar
  10. 10.
    Clark C, Fraser K, Hand S, Hansen J, Jul E, Limpach C, Pratt I, Warfield A (2005) Live migration of virtual machines. USENIX Association, Berkeley, pp 273–286 Google Scholar
  11. 11.
    Bunde D (2009) Power-aware scheduling for makespan and flow. J Sched 12(5):489–500 MathSciNetzbMATHCrossRefGoogle Scholar
  12. 12.
    von Laszewski G, Wang L, Younge A, He X (2009) Power-aware scheduling of virtual machines in dvfs-enabled clusters. In: IEEE, pp 1–10. doi: 10.1109/CLUSTR.2009.5289182 Google Scholar
  13. 13.
    Agarwal R, Gustavson F, Zubair M (2010) Exploiting functional parallelism of power2 to design high-performance numerical algorithms. IBM J Res Dev 38(5):563–576 CrossRefGoogle Scholar
  14. 14.
    Buyya R, Beloglazov A, Abawajy J (2010) Energy-efficient management of data center resources for cloud computing: a vision, architectural elements, and open challenges. In: International conference on parallel and distributed processing techniques and applications (PDPTA), Las Vegas, USA Google Scholar
  15. 15.
    Beloglazov A, Buyya R, Lee YC, Zomaya A (2010) A taxonomy and survey of energy-efficient data centers and cloud computing systems. Cloud Computing and Distributed Systems Laboratory. The University of Melbourne, Melbourne Google Scholar
  16. 16.
    Nathuji R, Schwan K (2007) Virtualpower: coordinated power management in virtualized enterprise systems. In: ACM, pp 265–278. doi: 10.1145/1294261.1294287 Google Scholar
  17. 17.
    Kusic D, Kephart J, Hanson J, Kandasamy N, Jiang G (2009) Power and performance management of virtualized computing environments via lookahead control. Clust Comput 12(1):1–15 CrossRefGoogle Scholar
  18. 18.
    Stillwell M, Schanzenbach D, Vivien F, Casanova H (2009) Resource allocation using virtual clusters. IEEE Computer Society Press, Los Alamitos, pp 260–267 Google Scholar
  19. 19.
    Song Y, Wang H, Li Y, Feng B, Sun Y (2009) Multi-tiered on-demand resource scheduling for vm-based data center. IEEE Computer Society Press, Los Alamitos, pp 148–155 Google Scholar
  20. 20.
    Cardosa M, Korupolu M, Singh A (2009) Shares and utilities based power consolidation in virtualized server environments. In: IEEE, pp 327–334. doi: 10.1109/INM.2009.5188832 Google Scholar
  21. 21.
    Verma A, Ahuja P, Neogi A (2008) Pmapper: Power and migration cost aware application placement in virtualized systems. Paper presented at the Proceedings of the 9th ACM/IFIP/USENIX international conference on middleware, Leuven, Belgium Google Scholar
  22. 22.
    Verma A, Dasgupta G, Nayak T, De P, Kothari R (2009) Server workload analysis for power minimization using consolidation. USENIX Association, Berkeley, p 28 Google Scholar
  23. 23.
    Srikantaiah S, Kansal A, Zhao F (2008) Energy-aware consolidation for cloud computing. USENIX Association, Berkeley, p 10 Google Scholar
  24. 24.
    Kim K, Beloglazov A, Buyya R (2009) Power-aware provisioning of cloud resources for real-time services. In: ACM, pp 1–6. doi: 10.1145/1657120.1657121 Google Scholar
  25. 25.
    Jang J, Jeon M, Kim H, Jo H, Kim J, Maeng S (2010) Energy reduction in consolidated servers through memory-aware virtual machine scheduling. IEEE Trans Comput. doi: 10.1109/TC.2010.82 Google Scholar
  26. 26.
    Tang Q, Gupta S, Varsamopoulos G (2008) Energy-efficient thermal-aware task scheduling for homogeneous high-performance computing data centers: a cyber-physical approach. IEEE Trans Parallel Distrib Syst 1458–1472. doi: 10.1109/TPDS.2008.111
  27. 27.
    Tang Q, Gupta S, Varsamopoulos G (2008) Thermal-aware task scheduling for data centers through minimizing heat recirculation. In: IEEE, pp 129–138. doi: 10.1109/CLUSTR.2007.4629225 Google Scholar
  28. 28.
    Kivity A, Kamay Y, Laor D, Lublin U, Liguori A (2007) Kvm: the Linux virtual machine monitor, pp 225–230 Google Scholar
  29. 29.
    Buyya R, Yeo CS, Venugopal S, Broberg J, Brandic I (2009) Cloud computing and emerging it platforms: vision, hype, and reality for delivering computing as the 5th utility. Future Gener Comput Syst 25(6):599–616. doi: 10.1016/j.future.2008.12.001 CrossRefGoogle Scholar
  30. 30.
    Sysbench benchmark suite (2010)
  31. 31.
    Hermenier F, Lorca X, Menaud J-M, Muller G, Lawall J (2009) Entropy: a consolidation manager for clusters. Paper presented at the Proceedings of the 2009 ACM SIGPLAN/SIGOPS international conference on virtual execution environments, Washington, DC, USA Google Scholar
  32. 32.
    Uhlig R, Neiger G, Rodgers D, Santoni A, Martins F, Anderson A, Bennett S, Kagi A, Leung F, Smith L (2005) Intel virtualization technology. Computer 38(5):48–56 CrossRefGoogle Scholar
  33. 33.
    Qemu open source processor emulator project (2010)
  34. 34.
    Wei J, Yisu Z, Yan C, Wei F, Yu C, Yuanchun S, Qingbo W (2009) Cfs optimizations to kvm threads on multi-core environment. In: Parallel and distributed systems (ICPADS), 2009 15th international conference on, 8–11 Dec, 2009, pp 348–354 Google Scholar
  35. 35.
    Ranganathan P, Leech P, Irwin D, Chase J (2006) Ensemble-level power management for dense blade servers. doi: 10.1109/ISCA.2006.20
  36. 36.
    Khanna G, Beaty K, Kar G, Kochut A (2006) Application performance management in virtualized server environments. Paper presented at the IEEE network operations and management symposium, Vancouver, BC Google Scholar
  37. 37.
    Bobroff N, Kochut A, Beaty K (2007) Dynamic placement of virtual machines for managing sla violations. In: IEEE, pp 119–128. doi: 10.1109/INM.2007.374776 Google Scholar
  38. 38.
    Khargharia B, Hariri S, Yousif MS (2008) Autonomic power and performance management for computing systems. Clust Comput 11(2):167–181 CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  1. 1.Distributed Systems Laboratory, School of Computer EngineeringIran University of Science and TechnologyTehranIran

Personalised recommendations