The Journal of Supercomputing

, Volume 66, Issue 3, pp 1629–1655 | Cite as

Analysis of virtual machine live-migration as a method for power-capping

  • Jinkyu Jeong
  • Sung-Hun Kim
  • Hwanju Kim
  • Joonwon Lee
  • Euiseong Seo


To reduce the construction cost of the power-supplying infrastructure in data centers and to increase the utilization of the existing one, many researchers have introduced software-based or hardware-based power-capping schemes. In servers with consolidated virtual machines, which can be easily found in cloud systems, exporting virtual machines to other light-loaded servers through live-migration is one of the key approaches to impose power-capping on servers. Up until now, most researchers who have tried to achieve power-capping through live-migration assumed that exporting a virtual machine instantly reduces the server power consumption. However, our analysis introduced in this paper reveals that the power consumption remains high or increases for a few seconds during a migration instance. This behavior contradicts the aim of power-capping, and may endanger the stability of servers. Based on this observation, we also propose and evaluate two power-suppressing live-migration schemes to resolve the power overshooting issue. Our evaluation shows that both approaches immediately limit the power consumption after live-migration is initiated.


Virtualization Live-migration Power-aware computing Cloud computing Distributed systems 



This research was supported by Next-Generation Information Computing Development Program (2012-0006423) and Basic Science Research Program (2012R1A1A2A10038823) through the National Research Foundation of Korea (NRF) funded by the Ministry of Education, Science, and Technology.


  1. 1.
    Ackaouy E (2006) The Xen Credit CPU scheduler. In: Proceedings of 2006 Fall Xen Summit Google Scholar
  2. 2.
    Armbrust M, Fox A, Griffith R, Joseph AD, Katz RH, Konwinski A, Lee G, Patterson DA, Rabkin A, Stoica I, Zaharia M (2009) Above the clouds: a Berkeley view of cloud computing. Technical report UCB/EECS-2009-28, UC Berkeley Google Scholar
  3. 3.
    Barham P, Dragovic B, Fraser K, Hand S, Harris T, Ho A, Neugebauer R, Pratt I, Warfield A (2003) Xen and the art of virtualization. In: Proceedings of the nineteenth ACM symposium on operating systems principles Google Scholar
  4. 4.
    Choi J, Govindan S, Jeong J, Urgaonkar B, Sivasubramaniam A (2010) Power consumption prediction and power-aware packing in consolidated environments. IEEE Trans Comput 59:1640–1654 MathSciNetCrossRefGoogle Scholar
  5. 5.
    Choi J, Govindan S, Urgaonkar B, Sivasubramaniam A (2008) Profiling, prediction, and capping of power consumption in consolidated environments. In: IEEE international symposium on modeling, analysis and simulation of computers and telecommunication systems, MASCOTS 2008. Google Scholar
  6. 6.
    Clark C, Fraser K, Hand S, Hansen JG, Jul E, Limpach C, Pratt I, Warfield A (2005) Live migration of virtual machines. In: Proceedings of the 2nd conference on symposium on networked systems design & implementation, NSDI’05, Berkeley, CA, USA. USENIX Association, Berkeley, pp 273–286 Google Scholar
  7. 7.
    Cochran R, Hankendi C, Coskun AK, Reda S (2011) Pack & cap: adaptive DVFS and thread packing under power caps. In: Proceedings of the 44th annual IEEE/ACM international symposium on microarchitecture Google Scholar
  8. 8.
    Das T, Padala P, Padmanabhan V, Ramjee R, Shin KG (2010) Litegreen: saving energy in networked desktops using virtualization. In: Proceedings of USENIX annual technical conference, Usenix ATC’10 Google Scholar
  9. 9.
    Fan X, Weber W-D, Barroso LA (2007) Power provisioning for a warehouse-sized computer. In: Proceedings of the 34th annual international symposium on computer architecture, ISCA’07 Google Scholar
  10. 10.
    Gandhi A, Harchol-Balter M, Das R, Kephart JO, Lefurgy C (2009) Power capping via forced idleness. In: Proceedings of workshop on energy-efficient design, WEED’09 Google Scholar
  11. 11.
    Gove D (2007) Cpu2006 working set size. Comput Archit News 35(1):90–96 CrossRefGoogle Scholar
  12. 12.
    Govindan S, Choi J, Urgaonkar B, Sivasubramaniam A, Baldini A (2009) Statistical profiling-based techniques for effective power provisioning in data centers. In: Proceedings of the 4th ACM European conference on computer systems, EuroSys’09. ACM, New York, pp 317–330 CrossRefGoogle Scholar
  13. 13.
    Henning J (2001) SPEC CPU2000 memory footprint.
  14. 14.
    Hewlett-Packard Development Company, LP (2011) HP power capping and HP dynamic power capping for ProLiant servers, 2nd edn. Technology brief TC110107TB, Hewlett-Packard Development Company, LP Google Scholar
  15. 15.
    Hines MR, Gopalan K (2009) Post-copy based live virtual machine migration using adaptive pre-paging and dynamic self-ballooning. In: Proceedings of the 2009 ACM SIGPLAN/SIGOPS international conference on virtual execution environments, VEE’09. ACM, New York, pp 51–60 CrossRefGoogle Scholar
  16. 16.
    Hirofuchi T, Nakada H, Itoh S, Sekiguchi S (2010) Enabling instantaneous relocation of virtual machines with a lightweight vmm extension. In: 10th IEEE/ACM international conference on cluster, cloud and grid computing, CCGrid’10, pp 73–83 CrossRefGoogle Scholar
  17. 17.
    Hirofuchi T, Nakada H, Itoh S, Sekiguchi S (2011) Making VM consolidation more energy-efficient by postcopy live migration. In: Proceedings of the second international conference on cloud computing, GRIDs, and virtualization Google Scholar
  18. 18.
    Huang Q, Gao F, Wang R, Qi Z (2011) Power consumption of virtual machine live migration in clouds. In: Third international conference on communications and mobile computing, CMC’11, pp 122–125 CrossRefGoogle Scholar
  19. 19.
    Jenkins D (2009) Intel intelligent power node manager. White paper 322705-002US, Intel Corporation Google Scholar
  20. 20.
    Jin H, Gao W, Wu S, Shi X, Wu X, Zhou F (2011) Optimizing the live migration of virtual machine by CPU scheduling. J Netw Comput Appl 34(4):1088–1096 CrossRefGoogle Scholar
  21. 21.
    Kansal A, Zhao F, Liu J, Kothari N, Bhattacharya AA (2010) Virtual machine power metering and provisioning. In: Proceedings of the 1st ACM symposium on cloud computing Google Scholar
  22. 22.
    Kozuch M, Satyanarayanan M (2002) Internet suspend/resume. In: Proceedings of the fourth IEEE workshop on mobile computing systems and applications Google Scholar
  23. 23.
    Lagar-Cavilla HA, Whitney JA, Scannell AM, Patchin P, Rumble SM, de Lara E, Brudno M, Satyanarayanan M (2009) Snowflock: rapid virtual machine cloning for cloud computing. In: Proceedings of the 4th ACM European conference on computer systems Google Scholar
  24. 24.
    Lee Y, Zomaya A (2012) Energy efficient utilization of resources in cloud computing systems. J Supercomput 60:268–280 CrossRefGoogle Scholar
  25. 25.
    Lefèvre L, Orgerie A-C (2010) Designing and evaluating an energy efficient cloud. J Supercomput 51:352–373 CrossRefGoogle Scholar
  26. 26.
    Lefurgy C, Wang X, Allen-Ware M (2007) Server-level power control. In: Proceedings of the fourth international conference on autonomic computing Google Scholar
  27. 27.
    Lefurgy C, Wang X, Allen-Ware M (2008) Power capping: a prelude to power shifting. Clust Comput 11:183–195 CrossRefGoogle Scholar
  28. 28.
    Lim H, Kansal A, Liu J (2011) Power budgeting for virtualized data centers. In: Proceedings of the 2011 USENIX conference on USENIX annual technical conference, USENIXATC’11. USENIX Association, Berkeley, p 5 Google Scholar
  29. 29.
    Liu H, Jin H, Liao X, Hu L, Yu C (2009) Live migration of virtual machine based on full system trace and replay. In: Proceedings of the 18th ACM international symposium on high performance distributed computing Google Scholar
  30. 30.
    Liu H, Xu C-Z, Jin H, Gong J, Liao X (2011) Performance and energy modeling for live migration of virtual machines. In: Proceedings of the 20th international symposium on high performance distributed computing, HPDC’11. ACM, New York, pp 171–182 Google Scholar
  31. 31.
    Mitchell-Jackson J, Koomey J, Nordman B, Blazek M (2003) Data center power requirements: measurements from silicon valley. Energy 28(8):837–850 CrossRefGoogle Scholar
  32. 32.
    Nathuji R, Schwan K (2008) VPM tokens: virtual machine-aware power budgeting in datacenters. In: Proceedings of the 17th international symposium on high performance distributed computing, HPDC’08, pp 119–128 CrossRefGoogle Scholar
  33. 33.
    Nelson M, Lim B-H, Hutchins G (2005) Fast transparent migration for virtual machines. In: Proceedings of Usenix annual technical conference, Usenix ATC’05 Google Scholar
  34. 34.
    Raghavendra R, Ranganathan P, Talwar V, Wang Z, Zhu X (2008) No “power” struggles: coordinated multi-level power management for the data center. In: Proceedings of the 13th international conference on architectural support for programming languages and operating systems, ASPLOS XIII. ACM, New York, pp 48–59 CrossRefGoogle Scholar
  35. 35.
    Rasmussen N (2010) Implementing energy efficient data centers. APC white paper, no 114, rev 1 Google Scholar
  36. 36.
    Sapuntzakis CP, Chandra R, Pfaff B, Chow J, Lam MS, Rosenblum M (2002) Optimizing the migration of virtual computers. Oper Syst Rev 36:377–390 CrossRefGoogle Scholar
  37. 37.
    Seo E, Jeong J, Park S, Lee J (2008) Energy efficient scheduling of real-time tasks on multicore processors. IEEE Trans Parallel Distrib Syst 19:1540–1552 CrossRefGoogle Scholar
  38. 38.
    Sharifi M, Salimi H, Najafzadeh M (2012) Power-efficient distributed scheduling of virtual machines using workload-aware consolidation techniques. J Supercomput 61:46–66 CrossRefGoogle Scholar
  39. 39.
    Verma A, Kumar G, Koller R, Sen A (2011) Cosmig: modeling the impact of reconfiguration in a cloud. In: Proceedings of the 2011 IEEE 19th annual international symposium on modelling, analysis, and simulation of computer and telecommunication systems, MASCOTS’11. IEEE Comput Soc, Washington, pp 3–11 CrossRefGoogle Scholar
  40. 40.
    Voorsluys W, Broberg J, Venugopal S, Buyya R (2009) Cost of virtual machine live migration in clouds: a performance evaluation. In: Proceedings of the 1st international conference on cloud computing Google Scholar
  41. 41.
    Wang Z, Zhu X, McCarthy C, Ranganathan P, Talwar V (2008) Feedback control algorithms for power management of servers. In: Proceedings of the third international workshop on feedback control implementation and design in computing systems and networks Google Scholar
  42. 42.
    Ware M, Rajamani K, Floyd M, Brock B, Rubio JC, Rawson F, Carter JB (2010) Architecting for power management: the IBM® Power7™ approach. In: Proceedings of the 16th international symposium on high performance computer architecture Google Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Jinkyu Jeong
    • 1
  • Sung-Hun Kim
    • 2
  • Hwanju Kim
    • 1
  • Joonwon Lee
    • 2
  • Euiseong Seo
    • 2
  1. 1.Department of Computer ScienceKorea Advanced Institute of Science and TechnologyDaejeonKorea
  2. 2.College of ICESungkyunkwan UniversitySuwonKorea

Personalised recommendations