Energy Efficiency

, Volume 10, Issue 2, pp 475–498 | Cite as

Methodology for energy aware adaptive management of virtualized data centers

  • Tudor Cioara
  • Ionut Anghel
  • Ioan Salomie
Original Article


This paper proposes a methodology for energy aware management of virtualized data centers (DC) based on dynamically adapting and scaling the computing capacity to the characteristics of the workload. To assess the energy efficiency of DC operation, we have defined a novel ontological model for representing its energy and performance characteristics and a new metric for aggregating Green and Key Performance Indicators and calculating at run-time the DC Greenness Level. Workload balancing and consolidation is achieved by means of an automated reinforcement learning-based decision process targeting to increase the workload density and to scale down the unused computing resources. Evaluation results show that up to 15.6 % energy savings are obtained on our test bed DC. Tests conducted in a simulated environment show that the time and space overhead of our methodology are within reasonable limits and that by organizing the servers in hierarchical clusters, the methodology can manage highly dynamic workload in large DCs with thousands of servers. The methodology is already implemented in the Green Cloud Scheduler, an official component of the OpenNebula Middleware which is available in the OpenNebula Ecosystem web site to be downloaded and used.


Energy efficient data centers Green management of computing resources Energy awareness Cloud computing Reinforcement learning 



This work has been partially funded by the GAMES project (2012) and has been partly funded by the European Commission ICT activity of the 7th Framework Program (number ICT-248514). This work expresses the opinions of the authors and not necessarily those of the European Commission. The European Commission is not liable for any use that may be made of the information contained in this work. This document is a collaborative effort. The scientific contribution of all authors is the same.

Compliance with ethical standards


This study was funded by GAMES FP7 project (grant number ICT-248,514).

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.


  1. America’s Data Centers Consuming and Wasting Growing Amounts of Energy. (2015).
  2. Industry Outlook. (2014). Data center energy efficiency. The Data Center Journal.
  3. Benik, A., & Ventures, B. (2013). The sorry state of server utilization and the impending post-hypervisor era.
  4. Zhang, Z, Hsu, C.-C., & Chang, M. (2015). Cool cloud: a practical dynamic virtual machine placement framework for energy aware data centers, IEEE 8th International Conference on Cloud Computing.Google Scholar
  5. Kayed, A., & Akijian, T. (2015). Resource allocation technique to obtain energy efficient cloud, ICEMIS’15, September 24–26, Istanbul, Turkey.Google Scholar
  6. Delforge, P., & Whitney, J. (2014). Data center efficiency assessment, scaling up energy efficiency across the data center industry: evaluating key drivers and barriers.
  7. Jones Jr., M. E., Wei, B. W. Y., & Hung, D. L. (2013). Laptop energy-saving opportunities based on user behaviors. Energy Efficiency, 6, 425–431. doi: 10.1007/s12053-012-9167-5.CrossRefGoogle Scholar
  8. Oikonomou, E., Panagiotou, D., & Rouskas, A. (2015). Energy-aware management of virtual machines in cloud data centers, 16th EANN workshops, September 25–28, Rhodes Island, Greece.Google Scholar
  9. Howard, A. J., & Holmes, J. (2012). Addressing data center efficiency: lessons learned from process evaluations of utility energy efficiency programs. Energy Efficiency, 5, 137–148 SPECIAL ISSUE – VINE.CrossRefGoogle Scholar
  10. Panagiotou, D., Oikonomou, E., & Rouskas, A. (2015). Energy-efficient virtual machine provisioning mechanism in cloud computing environments. Proceedings of the 19th Panhellenic Conference on Informatics (pp. 197–202).Google Scholar
  11. Liao, D., Li, K., Sun, G., Anand, V., Gong, Y., & Tan, Z. (2015). Energy and performance management in large data centers: a queuing theory perspective. International Conference on Computing, Networking and Communications (ICNC), Workshop on Computing, Networking and Communications (CNC).Google Scholar
  12. Lin, H., Qi, X., Yang, S., & Midkiff, S. P. (2015). Workload-driven VM consolidation in Cloud Data Center, IEEE 29th International Parallel and Distributed Processing Symposium.Google Scholar
  13. Ardagna, D., Panicucci, B., Trubian, M., & Zhang, L. (2012). Energy-aware autonomic resource allocation in multitier virtualized environments. Proceedings of IEEE Transactions on services computing, 5/1, 2–19. doi: 10.1109/TSC.2010.42.CrossRefGoogle Scholar
  14. García-Valls, M., Cucinotta, T., & Lu, C. (2014). Challenges in real-time virtualization and predictable cloud computing. Journal of Systems Architecture, 60(9), 726–740.CrossRefGoogle Scholar
  15. Khani, H., Latifi, A., Yazdani, N., & Mohammadi, S. (2015). Distributed consolidation of virtual machines for power efficiency in heterogeneous cloud data centers. Computers & Electrical Engineering, 47, 173–185.CrossRefGoogle Scholar
  16. Barbagallo, D., Di Nitto, E., Dubois, D., & Mirandola, R. (2010). A bio-inspired algorithm for energy optimization in a self-organizing data center, in proceedings first international conference on self-organizing architectures. LNCS, 6090, 127–151.Google Scholar
  17. Kansal, N. J., & Chana, I. (2015). Artificial bee colony based energy-aware resource utilization technique for cloud computing. Concurrency and Computation: Practice and Experience, 27(5), 1207–1225.CrossRefGoogle Scholar
  18. García-Valls, M., Rodriguez-Lopez, I., & Fernandez-Villar, L. (2013). iLAND: an enhanced middleware for real-time reconfiguration of service oriented distributed real-time systems. IEEE Transactions on Industrial Informatics, 9(1), 228–336.CrossRefGoogle Scholar
  19. Csorba, M., Meling, H., & Heegaard, P. (2010). Ant system for service deployment in private and public clouds. In Proceedings of the 2nd workshop on Bio-inspired algorithms for distributed systems. Washington, USA, pp. 19–28. doi: 10.1145/1809018.1809024.
  20. Jeyarani, R., Nagaveni, N., & Ram, R. V. (2012). Design and implementation of adaptive power-aware virtual machine provisioner (APA-VMP) using swarm intelligence. In Future Generation Computer Systems, Elsevier, 28(5), 811–821. doi: 10.1016/j.future.2011.06.002.CrossRefGoogle Scholar
  21. Khan, A., Yan, X., Tao, S., & Anerousis, N. (2012). Workload characterization and prediction in the cloud: a multiple time series approach. In Network Operations and Management Symposium (NOMS), IEEE, Hawaii, USA, pp. 1287–1294. doi: 10.1109/NOMS.2012.6212065.
  22. Ardito, L., & Morisio, M. (2013). Green IT—available data and guidelines for reducing energy consumption in IT systems, Sustainable Computing: Informatics and Systems, Elsevier. Available online.
  23. Huai, W., Huang, W., Jin, S., & Qian, Z. (2015). Towards energy efficient scheduling for online tasks in cloud data centers based on DVFS. 9th International Conference on Innovative Mobile and Internet Services in Ubiquitous Computing.Google Scholar
  24. Hieu, N. T., Di Francesco, M., & Ylä-Jääski, A. (2015). Virtual machine consolidation with usage prediction for energy-efficient cloud data centers. IEEE 8th International Conference on Cloud Computing.Google Scholar
  25. FIT4GREEN FP7 Project. (2012).
  26. Mondal, S. K., & Muppala, J. K. (2014). Energy modeling of virtual machine replication schemes with checkpointing in data centers. IEEE Fourth International Conference on Big Data and Cloud Computing.Google Scholar
  27. Borgetto, D., Casanova, H., Da Costa, G., & Pierson, J.-M. (2012). Energy-aware service allocation. Future Generation Computer Systems, 28(5), 769–779. doi: 10.1016/j.future.2011.04.018.CrossRefGoogle Scholar
  28. Garg, S. K., Yeo, C. S., Anandasivam, A., & Buyya, R. (2011). Environment-conscious scheduling of HPC applications on distributed cloud-oriented data centers. Journal of Parallel Distributed Computing, Elsevier, 71(6), 732–749. doi: 10.1016/j.jpdc.2010.04.004.CrossRefzbMATHGoogle Scholar
  29. Cioara, T., Anghel, I., Salomie, I., Copil, G., Moldovan, D., & Pernici, B. (2011). A context aware self-adapting algorithm for managing the energy efficiency of IT service centres. Ubiquitous Computing and Communication Journal, 6 (1).Google Scholar
  30. Salomie, I., Cioara, T., Anghel, I., & Dinsoreanu, M. (2008). RAP—a basic context awareness model. In Proceedings of the 4th International Conference on Intelligent Computer Communication and Processing. IEEE, pp. 315–318. doi: 10.1109/ICCP.2008.4648395.
  31. Pernici, B., Cappiello, C., Fugini, M. G., Plebani, P., Vitali, M., Salomie, I., Cioara, T., & Anghel, I. (2012). Setting energy efficiency goals in data centres: the GAMES approach. Lecture Notes in Computer Science., 7396, 1–12.CrossRefGoogle Scholar
  32. Nagios. (2016). The industry standard in IT infrastructure monitoring.
  33. Kipp, A., Liu, J., Jiang, T., Buchholz, J., Schubert, L., Berge, M., & Christmann, W. (2011). Testbed architecture for generic, energy-aware evaluations and optimisations. Proc. of the First International Conference on Advanced Communications and Computation, Spain.Google Scholar
  34. CLM5. (2012). Christ-Elektronik Power Meter.
  35. OpenNebula Cloud Data Center Management Solution. (2016).
  36. KVM. (2012). Kernel Based Virtual Machine.
  37. OpenSSH. (2012). Free SSH implementation.
  38. Wake-On-LAN. (2012).
  39. TPC-Energy Specification. (2012).
  40. Spec. (2008). Standard Performance Evaluation Corporation,
  41. Stanley, J., Brill, K., & Koomey, J. (2009). Four metrics define data center greenness. Uptime Institute White paper.
  42. Protégé. (2012). Protégé home.
  43. KAON2 Introduction. (2012).
  44. HSQLDB. (2012). 100 % Java Database.
  45. Hibernate persistence. (2012).
  46. Prevayler. (2012). Prevayler API documentation.
  47. Pellet. (2012). Pellet Features.
  48. Reasoners and rule engines: Jena inference support. (2012).
  49. SWRL. (2012). A semantic web rule language combining OWL and rule ML.
  50. Salehie, M., & Tahvildari, L. (2009). Self-adaptive software: landscape and research challenges. ACM Transactions on Autonomous and Adaptive Systems, TAAS, 4, 2. doi: 10.1145/1516533.1516538.Google Scholar
  51. Johnson, P., & Marker, T. (2009). Data centre energy efficiency product profile report No 2009/05, Prepared for Equipment Energy Efficiency Committee.
  52. GAMES project. (2012). Green active management of energy in IT service centers.
  53. GEYSER project. (2016). Green networked data centers as energy prosumers in smart city environments.
  54. OpenNebula Green Cloud Scheduler. (2012).
  55. Environmentally sustainable data centre for Smart Cities. (2016).
  56. Valls, M. G., Alonso, A., & de la Puente, J. A. (2012). A dual-band priority assignment algorithm for dynamic QoS resource management. Future Generation Computer Systems, 28(6), 902–912.CrossRefGoogle Scholar
  57. Dupont, C., Giuliani, G., Hermenier, F., Schulze, T., & Somov, A. (2012). An energy aware framework for virtual machine placement in cloud federated data centres. Proceeding of 3rd International Conference on Future Energy Systems: where energy, computing and communication meet.Google Scholar
  58. Lent, R. (2013). A model of a network server performance and power consumption. Sustainable Computing: Informatics and Systems, 3(2).Google Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2016

Authors and Affiliations

  1. 1.Computer Science DepartmentTechnical University of Cluj-NapocaCluj-NapocaRomania

Personalised recommendations