Journal of Heuristics

, Volume 25, Issue 3, pp 339–375 | Cite as

Virtual machine consolidation using constraint-based multi-objective optimization

  • Miguel Terra-NevesEmail author
  • Inês Lynce
  • Vasco Manquinho


With the blooming of cloud computing, the demand for data centers has been rising greatly in recent years. Their energy consumption and environmental impact has become much more significant due to the continuous growth of data center supply. It is possible to reduce the amount of energy consumed by a data center by shutting down unnecessary servers and maintaining only a subset running, such that it is enough to fulfill the resource demand. With recent advances in virtualization technology, it even became possible to consolidate the workload of multiple under-utilized servers into a single server. However, too aggressive consolidation may lead to significant degradation of data center performance. Therefore, the problem of simultaneously minimizing energy consumption and performance degradation in a data center is a complex and challenging problem. In this paper, a novel multi-objective Boolean optimization encoding for virtual machine consolidation is proposed and several approaches to solve it are described and compared. Moreover, this encoding is extended to consider anti-collocation constraints and the migration of virtual machines that are initially placed. This work is in part motivated by the great improvements in the performance of Boolean optimization solvers, thus increasing their applicability and effectiveness for a wider spectrum of complex problems. In this case, specific techniques are applied to further boost the performance, namely search space reduction by symmetry breaking and heuristic reduction of the instance size. An extensive experimental evaluation shows the suitability of the proposed solution in comparison to the state-of-the-art approaches based on stochastic methods.


Virtual machine consolidation Multi-objective optimization Pseudo-Boolean optimization Live migration Cloud computing 



The authors would like to thank Professors Salvador Pinto Abreu and Pedro Salgueiro from Universidade de Évora for granting the authors permission to use their cluster. This work was supported by national funds through Fundação para a Ciência e a Tecnologia (FCT) with references UID/CEC/50021/2013, SFRH/BD/111471/2015, CMU/AIR/0022/2017 and PTDC/CCI-COM/31198/2017.


  1. Aloul, F.A., Sakallah, K.A., Markov, I.L.: Efficient symmetry breaking for Boolean satisfiability. IEEE Trans. Comput. 55(5), 549–558 (2006)Google Scholar
  2. Bartók D., Mann ZÁ.: A branch-and-bound approach to virtual machine placement. In: HPI Cloud Symposium Operating the Cloud, pp. 49–63 (2015)Google Scholar
  3. Beloglazov, A., Buyya, R.: Optimal online deterministic algorithms and adaptive heuristics for energy and performance efficient dynamic consolidation of virtual machines in cloud data centers. Concurr. Comput. Pract. Exp. 24(13), 1397–1420 (2012)Google Scholar
  4. Beloglazov, A., Abawajy, J.H., Buyya, R.: Energy-aware resource allocation heuristics for efficient management of data centers for cloud computing. Future Gener. Comput. Syst. 28(5), 755–768 (2012)Google Scholar
  5. Bjørner N., Phan A., Fleckenstein L.: \(\nu \)z: an optimizing SMT solver. In: 21st International Conference on Tools and Algorithms for the Construction and Analysis of Systems, TACAS, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS, London, pp. 194–199 (2015)Google Scholar
  6. Bobroff, N., Kochut, A., Beaty, K.A.: Dynamic placement of virtual machines for managing SLA violations. In: 10th IFIP/IEEE International Symposium on Integrated Network Management, Munich, pp. 119–128 (2007)Google Scholar
  7. Boros, E., Hammer, P.L.: Pseudo-Boolean optimization. Discrete Appl. Math. 123(1), 155–225 (2002)MathSciNetzbMATHGoogle Scholar
  8. Cambazard, H., Mehta, D., O’Sullivan, B., Simonis, H.: Constraint programming based large neighbourhood search for energy minimisation in data centres. In: 10th International Conference on Economics of Grids, Clouds, Systems, and Services GECON, Zaragoza, pp. 44–59 (2013)Google Scholar
  9. Carter, L., Wegman, M.N.: Universal classes of hash functions (extended abstract). In: 9th Annual ACM Symposium on Theory of Computing, Boulder, pp. 106–112 (1977)Google Scholar
  10. Chakraborty, S., Meel, K.S., Vardi, M.Y.: A scalable and nearly uniform generator of SAT witnesses. In: 25th International Conference on Computer Aided Verification CAV, Saint Petersburg, pp. 608–623 (2013)Google Scholar
  11. Chakraborty, S., Meel, K.S., Vardi, M.Y.: Algorithmic improvements in approximate counting for probabilistic inference: From linear to logarithmic SAT calls. In: 25th International Joint Conference on Artificial Intelligence, IJCAI, New York, pp. 3569–3576 (2016)Google Scholar
  12. Deb, K., Jain, H.: An evolutionary many-objective optimization algorithm using reference-point-based nondominated sorting approach, part I: solving problems with box constraints. IEEE Trans. Evolut. Comput. 18(4), 577–601 (2014)Google Scholar
  13. Deb, K., Agrawal, S., Pratap, A., Meyarivan, T.: A fast elitist non-dominated sorting genetic algorithm for multi-objective optimisation: NSGA-II. In: 6th International Conference on Parallel Problem Solving from Nature PPSN, Paris, pp. 849–858 (2000)Google Scholar
  14. Eén, N., Sörensson, N.: Translating pseudo-Boolean constraints into SAT. J. Satisf. Boolean Model. Comput. 2(1–4), 1–26 (2006)zbMATHGoogle Scholar
  15. Fan, X., Weber, W., Barroso, L.A.: Power provisioning for a warehouse-sized computer. In: 34th International Symposium on Computer Architecture, ISCA, San Diego, pp. 13–23 (2007)Google Scholar
  16. Ferreto, T.C., Netto, M.A.S., Calheiros, R.N., Rose, C.A.F.D.: Server consolidation with migration control for virtualized data centers. Future Gener. Comput. Syst. 27(8), 1027–1034 (2011)Google Scholar
  17. Fox, A., Griffith, R., Joseph, A., Katz, R., Konwinski, A., Lee, G., Patterson, D., Rabkin, A., Stoica, I.: Above the clouds: a berkeley view of cloud computing. Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, Rep UCB/EECS, vol. 28, p. 13 (2009)Google Scholar
  18. Gao, Y., Guan, H., Qi, Z., Hou, Y., Liu, L.: A multi-objective ant colony system algorithm for virtual machine placement in cloud computing. J. Comput. Syst. Sci. 79(8), 1230–1242 (2013)MathSciNetGoogle Scholar
  19. Gomes, C.P., Sabharwal, A., Selman, B.: Near-uniform sampling of combinatorial spaces using XOR constraints. In: 20th Annual Conference on Neural Information Processing Systems on Advances in Neural Information Processing Systems, vol. 19, Vancouver, pp. 481–488 (2006)Google Scholar
  20. Graça, A., Lynce, I., Marques-Silva, J., Oliveira, A.: Efficient and accurate haplotype inference by combining parsimony and pedigree information. In: Algebraic and Numeric Biology. Springer, LNCS, vol. 6479, pp. 38–56 (2010)Google Scholar
  21. Guazzone, M., Anglano, C., Canonico, M.: Exploiting VM migration for the automated power and performance management of green cloud computing systems. In: 1st International Workshop on Energy Efficient Data Centers E2DC, Revised Selected Papers, Madrid, pp. 81–92 (2012)Google Scholar
  22. Guenter B., Jain N., Williams, C.: Managing cost, performance, and reliability tradeoffs for energy-aware server provisioning. In: INFOCOM 2011, 30th IEEE International Conference on Computer Communications, Joint Conference of the IEEE Computer and Communications Societies, Shanghai, pp. 1332–1340 (2011)Google Scholar
  23. Hermenier, F., Lorca, X., Menaud, J., Muller, G., Lawall, J.L.: Entropy: a consolidation manager for clusters. In: 5th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments VEE, Washington, pp. 41–50 (2009)Google Scholar
  24. Ignatiev, A., Janota, M., Marques-Silva, J.: Towards efficient optimization in package management systems. In: International Conference on Software Engineering, ACM, pp. 745–755 (2014)Google Scholar
  25. Jackson, D., Estler, H., Rayside, D., et al: The guided improvement algorithm for exact, general-purpose, many-objective combinatorial optimization. Technical Report MIT-CSAIL-TR-2009-033, MIT Massachusetts Institute of Technology (2009)Google Scholar
  26. Jo, C., Gustafsson, E., Son, J., Egger, B.: Efficient live migration of virtual machines using shared storage. In: ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments (co-located with ASPLOS) VEE, Houston, pp. 41–50 (2013)Google Scholar
  27. Jung, G., Hiltunen, M.A., Joshi, K.R., Schlichting, R.D., Pu, C.: Mistral: Dynamically managing power, performance, and adaptation cost in cloud infrastructures. In: International Conference on Distributed Computing Systems ICDCS, Genova, pp. 62–73 (2010)Google Scholar
  28. Le Berre, D., Parrain, A.: The sat4j library, release 2.2. J. Satisf. Boolean Model. Comput. 7(2–3), 59–60 (2010)Google Scholar
  29. Leavitt, N.: Is cloud computing really ready for prime time? Computer 1, 15–20 (2009)Google Scholar
  30. Li, R., Zheng, Q., Li, X., Wu, J.: A novel multi-objective optimization scheme for rebalancing virtual machine placement. In: IEEE International Conference on Cloud Computing CLOUD, pp. 710–717 (2016)Google Scholar
  31. Lin, W., Xu, S., Li, J., Xu, L., Peng, Z.: Design and theoretical analysis of virtual machine placement algorithm based on peak workload characteristics. Soft Comput. 21(5), 1301–1314 (2017)zbMATHGoogle Scholar
  32. Manquinho, V.M., Marques-Silva, J.P.: On using cutting planes in pseudo-Boolean optimization. J. Satisf. Boolean Model. Comput. 2(1–4), 209–219 (2006)zbMATHGoogle Scholar
  33. Marques-Silva, J., Lynce, I., Manquinho, V.M.: Symmetry breaking for maximum satisfiability. In: 15th International Conference on Logic for Programming, Artificial Intelligence, and Reasoning LPAR, Doha, pp. 1–15 (2008)Google Scholar
  34. Marques-Silva, J., Lynce, I., Malik, S.: Conflict-driven clause learning SAT solvers. In: Biere, A., Heule, M., Van Maaren, H., Walsh, T. (eds.) Handbook of Satisfiability, pp. 131–153 (2009)Google Scholar
  35. McMillan, K.L.: Applying SAT methods in unbounded symbolic model checking. In: 14th International Conference on Computer Aided Verification CAV, Copenhagen, pp. 250–264 (2002)Google Scholar
  36. Mills M.P.: The cloud begins with coal. Digital Power Group (2013). Accessed 31 Oct 2018
  37. Miranda, M., Lynce, I., Manquinho, V.: Inferring phylogenetic trees using pseudo-Boolean optimization. AI Commun. 27(3), 229–243 (2014)MathSciNetzbMATHGoogle Scholar
  38. Mishra, M., Sahoo, A.: On theory of VM placement: anomalies in existing methodologies and their mitigation using a novel vector based approach. In: IEEE International Conference on Cloud Computing CLOUD, Washington, pp. 275–282 (2011)Google Scholar
  39. Morgado, A., Marques-Silva, J.: Good learning and implicit model enumeration. In: 17th IEEE International Conference on Tools with Artificial Intelligence ICTAI, pp. 131–136 (2005)Google Scholar
  40. Nejad, M.M., Mashayekhy, L., Grosu, D.: Truthful greedy mechanisms for dynamic virtual machine provisioning and allocation in clouds. IEEE Trans. Parallel Distrib. Syst. 26(2), 594–603 (2015)Google Scholar
  41. Pareto, V.: Manuale di economia politica, vol. 13. Societa Editrice (1906)Google Scholar
  42. Portaluri, G., Giordano, S.: Multi objective virtual machine allocation in cloud data centers. In: IEEE International Conference on Cloud Networking Cloudnet, pp. 107–112 (2016)Google Scholar
  43. Prestwich, S., Quirke, C.: Boolean and pseudo-Boolean models for scheduling. In: International Workshop on Modelling and Reformulating Constraint Satisfaction Problems (2003)Google Scholar
  44. Ravi, K., Somenzi, F.: Minimal satisfying assignments for conjunctive normal formulae. In: International Conference on Tools and Algorithms for the Construction and Analysis of Systems (2004)Google Scholar
  45. Ribas, B.C., Suguimoto, R.M., Montaño, R.A.N.R., Silva, F., Castilho, M.A.: PBFVMC: A new pseudo-Boolean formulation to virtual-machine consolidation. In: Brazilian Conference on Intelligent Systems BRACIS, Fortaleza, pp. 201–206 (2013)Google Scholar
  46. Ruan, X., Chen, H.: Performance-to-power ratio aware virtual machine (VM) allocation in energy-efficient clouds. In: IEEE International Conference on Cluster Computing, CLUSTER, pp. 264–273 (2015)Google Scholar
  47. Saber, T., Ventresque, A., Gandibleux, X., Murphy, L.: Genepi: A multi-objective machine reassignment algorithm for data centres. In: 9th International Workshop on Hybrid Metaheuristics, pp. 115–129 (2014)Google Scholar
  48. Saber, T., Ventresque, A., Marques-Silva, J., Thorburn, J., Murphy, L.: MILP for the multi-objective VM reassignment problem. In: 27th International Conference on Tools with Artificial Intelligence, pp. 41–48 (2015)Google Scholar
  49. Saber, T., Marques-Silva, J., Thorburn, J., Ventresque, A.: Exact and hybrid solutions for the multi-objective VM reassignment problem. Int. J. Artif. Intell. Tools 26(1), 1–36 (2017)Google Scholar
  50. Sakallah, K.A.: Symmetry and satisfiability. Handb Satisf 185, 289–338 (2009)Google Scholar
  51. Salfner, F., Tröger, P., Polze, A.: Downtime analysis of virtual machine live migration. In: 4th International Conference on Dependability (DEPEND), IARIA, pp. 100–105 (2011)Google Scholar
  52. Sheini, H.M., Sakallah, K.A.: Pueblo: a hybrid pseudo-Boolean SAT solver. J. Satisf. Boolean Model. Comput. 2(1–4), 165–189 (2006)zbMATHGoogle Scholar
  53. Shi, L., Furlong, J., Wang, R.: Empirical evaluation of vector bin packing algorithms for energy efficient data centers. In: IEEE Symposium on Computers and Communications. ISCC, Split, pp. 9–15 (2013)Google Scholar
  54. Song, W., Xiao, Z., Chen, Q., Luo, H.: Adaptive resource provisioning for the cloud using online bin packing. IEEE Trans. Comput. 63(11), 2647–2660 (2014)MathSciNetzbMATHGoogle Scholar
  55. Srikantaiah, S., Kansal, A., Zhao, F.: Energy aware consolidation for cloud computing. In: Conference on Power Aware Computing and Systems, vol. 10, San Diego, pp. 1–5 (2008)Google Scholar
  56. Strunk, A.: Costs of virtual machine live migration: a survey. In: 8th IEEE World Congress on Services SERVICES, Honolulu, pp. 323–329 (2012)Google Scholar
  57. Terra-Neves, M., Lynce, I., Manquinho, VM.: Introducing pareto minimal correction subsets. In: 20th International Conference on Theory and Applications of Satisfiability Testing, pp. 195–211 (2017)Google Scholar
  58. Vaquero, L.M., Rodero-Merino, L., Caceres, J., Lindner, M.: A break in the clouds: towards a cloud definition. ACM SIGCOMM Comput. Commun. Rev. 39(1), 50–55 (2008)Google Scholar
  59. Verma, A., Ahuja, P., Neogi, A.: PMAPPER: power and migration cost aware application placement in virtualized systems. In: 9th ACM/IFIP/USENIX International Middleware Conference, Middleware, pp. 243–264 (2008)Google Scholar
  60. Verma, M., Gangadharan, G.R., Narendra, N.C., Ravi, V., Inamdar, V., Ramachandran, L., Calheiros, R.N., Buyya, R.: Dynamic resource demand prediction and allocation in multi-tenant service clouds. Concurr. Comput. Pract. Exp. 28(17), 4429–4442 (2016)Google Scholar
  61. Wang, J.V., Fok, K., Cheng, C., Tse, C.K.: A stable matching-based virtual machine allocation mechanism for cloud data centers. In: IEEE World Congress on Services, SERVICES, pp. 103–106 (2016)Google Scholar
  62. Xu, B., Peng, Z., Xiao, F., Gates, A.M., Yu, J.: Dynamic deployment of virtual machines in cloud computing using multi-objective optimization. Soft Comput. 19(8), 2265–2273 (2015)Google Scholar
  63. Xu, J., Fortes, J.A.B.: Multi-objective virtual machine placement in virtualized data center environments. In: IEEE/ACM International Conference on Green Computing and Communications, GreenCom, & International Conference on Cyber, Physical and Social Computing, CPSCom, Hangzhou, pp. 179–188 (2010)Google Scholar
  64. Zhang, Q., Li, H.: MOEA/D: a multiobjective evolutionary algorithm based on decomposition. IEEE Trans Evolut. Comput. 11(6), 712–731 (2007)Google Scholar
  65. Zheng, Q., Li, R., Li, X., Shah, N., Zhang, J., Tian, F., Chao, K., Li, J.: Virtual machine consolidated placement based on multi-objective biogeography-based optimization. Future Gener. Comput. Syst. 54, 95–122 (2016)Google Scholar
  66. Zitzler, E., Thiele, L.: Multiobjective evolutionary algorithms: a comparative case study and the strength pareto approach. IEEE Trans. Evolut. Comput. 3(4), 257–271 (1999)Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.INESC-ID / Instituto Superior TécnicoUniversidade de LisboaLisbonPortugal

Personalised recommendations