An adaptive overload threshold selection process using Markov decision processes of virtual machine in cloud data center

  • Zhihua Li


In Cloud Computing (CC), the cost for computation and energy is less by current cloud data centers because it exploits virtualization for an effective resource management. The Virtual Machine (VM) migration authorizes virtualization because it mitigates the difficulties of dynamic workload by repositioning VMs within cloud data centers. Through VM migration many goals of resource management are attained like load balancing, power management, fault tolerance, and system maintenance. The overload threshold is one of the key criterions to determine whether a host is overloaded or not. Achieving desired balance in guaranteeing quality of service, improving resource utilization and degrading energy consumption in data centers is the expected results of any overload threshold selection strategies. But, it is difficult due to the stochastic resource demands of VMs. In this paper, to address this problem, the overload threshold selection is modelled as a Markov decision process. With the solution of the improved Bellman optimality equation by the value iteration method, the optimization model is resolved, and the optimum overload threshold is adaptively selected. The hybrid processes are summarized as the Markov decision processes based adaptive overload threshold selection algorithm. Validations and comparisons are performed to illustrate its effectiveness and efficiency.


Host overload detection Markov decision processes Overload threshold Optimization model Value iteration method 



This work is supported by the Future Research Projects Funds for the Science and Technology Department of Jiangsu Province (Grant No. BY2013015-23) and the Fundamental Research Funds for the Ministry of Education (Grant No. JUSRP211A 41).


  1. 1.
  2. 2.
    Amazon Web Service,
  3. 3.
    Shen, Z., Subbiah, S., Gu, X., Wilkes, J.: Cloudscale: elastic resource scaling for multi-tenant cloud systems. In: Proceedings of the 2nd ACM Symposium on Cloud Computing, ACM, pp. 1–14 (2011)Google Scholar
  4. 4.
    Birke, R., Chen, L.Y., Smirni, E.: IEEE Proceedings of IEEE Data Centers in the Cloud: A Large Scale Performance Study, In: Proceedings of the 5th International Conference on Cloud Computing, pp. 336–343 (2013)Google Scholar
  5. 5.
    Gandhi, A., Harchol-Balter, M., Das, R., et al.: Optimal power allocation in server farms. ACM SIGMETRICS Perform. Eval. Rev. 37(1), 157–168 (2009)Google Scholar
  6. 6.
    Wood, T., Shenoy, P., Venkataramani, A., Yousif, M.: Sandpoper: black-box and gray-box resource management for virtual machines. Comput. Netw. 53(17), 2923–2938 (2009)CrossRefzbMATHGoogle Scholar
  7. 7.
    Zhu, Q., Zhu, J., Agrawal, G.: Power-aware consolidation of scientific workflows in virtualized environments. In: Proceedings of the 2010 ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis, pp. 1–12. IEEE Computer Society (2010)Google Scholar
  8. 8.
    Yu, L., Chen, L., Cai, Z., Shen, H., Liang, Y., Pan, Y.: Stochastic load balancing for virtual resource management in data center. IEEE Trans. Cloud Comput. (in press)Google Scholar
  9. 9.
    Clark, C., Fraser, K., Hand, S., Hansen, G.J., Jul, E., Limpach. C., et al.: Live migration of virtual machines. In: Proceedings of the 2Nd Conference on Symposium on Networked Systems Design & Implementation, vol. 2, pp. 273–286 (2005)Google Scholar
  10. 10.
    Xu, F., Liu, F., Liu, L., Jin, H., Li, B., Li, B.: iAware: making live migration of virtual machines interference-aware in the cloud. IEEE Trans. Comput. 63(12):3012–3025 (2014)Google Scholar
  11. 11.
    Beloglazov, A., Abawajy, J., Buyya, R.: Energy-aware resource allocation heuristics for efficient management of data centers for cloud computing. Future Gener. Comput. Syst. 28(5), 755–768 (2012)CrossRefGoogle Scholar
  12. 12.
    Farahnakian, F., Pahikkala, T., Liljeberg, P., Plosila, J.: Energy aware consolidation algorithm based on K-nearest neighbor regression for data centers. In: Proceedings of IEEE Utility and Cloud Computing (UCC), the 6th International Conference, ACM, p p. 256–259 (2013)Google Scholar
  13. 13.
    Farahnakian, F., Pahikkala, T., Liljeberg, P., Plosila, J., Tenhunen, H.: Utilization prediction aware VM consolidation approach for green cloud computing. In: IEEE Proceedings of the 8th International Conference on Cloud Computing, pp. 381–388 (2015)Google Scholar
  14. 14.
    Shaw, S.B., Singh, A.K.: Use of proactive and reactive hotspot detection technique to reduce the number of virtual machine migration and energy consumption in data center. Comput. Electr. Eng. 47, 241–254 (2015)CrossRefGoogle Scholar
  15. 15.
    Beloglazov, A., Buyya, R.: Optimal online deterministic algorithms and adaptive heuristics for energy and performance efficient dynamic consolidation of virtual machines in data centers. Concurr. Comput. 24(13), 1397–1420 (2012)CrossRefGoogle Scholar
  16. 16.
    Farahnakian, F., Liljeberg, P., Plosila, J.: LiRCUP: linear regression based CPU usage prediction algorithm for live migration of virtual machines in data centers. In: IEEE Proceedings of the 39th Euromicro Conference on Software Engineering and Advanced Applications, pp. 357–364 (2013)Google Scholar
  17. 17.
    Hieu, N.T., Di Francesco, M., Ylä-Jääski, A.: Virtual machine consolidation with usage prediction for energy-efficient data centers. In: IEEE Proceedings of the 8th International Conference on Cloud Computing, pp. 750–757 (2015)Google Scholar
  18. 18.
    Masoumzadeh, S.S., Hlavacs, H.: An intelligent and adaptive threshold-based schema for energy and performance efficient dynamic VM consolidation. In: Proceedings of European Conference on Energy Efficiency in Large Scale Distributed Systems, pp. 85–97 (2013)Google Scholar
  19. 19.
    Masoumzadeh, S.S., Hlavacs, H.: Dynamic virtual machine consolidation: a multi agent learning approach. In: IEEE Proceedings of the International Conference on Autonomic Computing, pp. 161–162 (2015)Google Scholar
  20. 20.
    Masoumzadeh, S.S., Hlavacs, H.: A cooperative multi agent learning approach to manage physical host nodes for dynamic consolidation of virtual machines. In: IEEE Proceedings of the Fourth Symposium on Network Cloud Computing and Applications, pp. 43–50 (2015)Google Scholar
  21. 21.
    Beloglazov, A., Buyya, R.: Managing overloaded hosts for dynamic consolidation of virtual machines in data centers under quality of service constraints. IEEE Trans. Parallel Distrib. Syst. 24(7), 1366–1379 (2013)CrossRefGoogle Scholar
  22. 22.
    Hermenier, F., Lorca, X., Menaud, J.M., Muller, G., Lawall, J.: Entropy: a consolidation manager for clusters. In: Proceeding of the 2009 ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments, ACM, 2009, pp. 41–50.Google Scholar
  23. 23.
    Farahnakian, F., Ashraf, A., Pahikkala, T., Liljeberg, P., Juha, Plosila, Porres, I., et al.: Using ant colony system to consolidate VMs for green cloud computing. IEEE Trans. Serv. Comput. 8(2), 187–198 (2015)CrossRefGoogle Scholar
  24. 24.
    Mann, Z.Á.: Rigorous results on the effectiveness of some heuristics for the consolidation of virtual machines in a data center. Future Gener. Comput. Syst. 51, 1–6 (2015)CrossRefGoogle Scholar
  25. 25.
    Chen, L., Shen, H., Sapra, K.: Distributed autonomous virtual resource management in data center using finite-markov decision process. In: Proceedings of the Symposium on Cloud Computing, ACM, pp. 1–13 (2014)Google Scholar
  26. 26.
    Feller, E., Morin, C., Esnault, A.: A case for fully decentralized dynamic VM consolidation in clouds. In: Proceeding of the 4th International Conference on Cloud Computing Technology and Science, IEEE, pp. 26–33 (2012)Google Scholar
  27. 27.
    Feller, E., Rilling, L., Morin, C.: Energy-aware ant colony based workload placement in clouds. In: Proceeding of the 12th International Conference on Grid Computing, IEEE/ACM, pp. 26–33 (2011)Google Scholar
  28. 28.
    Kaaouache, M.A., Bouamama, S.: Solving bin packing problem with a hybrid genetic algorithm for VM placement in cloud. Procedia Comput. Sci. 60(1), 1061–1069 (2015)CrossRefGoogle Scholar
  29. 29.
    Arjona, J.A., Anta, A.F., Ndez, A.A., Mosteiro M.A., Thraves C., Wang L.: Power-efficient assignment of virtual machines to PMs. Future Gener. Comput. Syst. 54(C):82–94 (2016)Google Scholar
  30. 30.
    Lago, D.G., Madeira, E.R.M, Bittencourt, L.F.: Power-aware virtual machine scheduling on clouds using active cooling control and DVFS. In: Proceeding of the 9th International Workshop on Middleware for Grids, ACM, vol. 2, pp. 1–6 (2011)Google Scholar
  31. 31.
    Guazzone, M., Anglano, C., Canonico, M.: Exploiting VM migration for the automated power and performance management of green cloud computing systems. In: Proceeding of International Workshop on Energy Efficient Data Centers. Springer Berlin Heidelberg, pp. 81–92 (2012)Google Scholar
  32. 32.
    Kusic, D., Kephart, J.O., Hanson, J.E., Kandasamy, N., Jiang, G.: Power and performance management of virtualized computing environments via lookahead control. Clust. Comput. 12(1), 1–15 (2009)CrossRefGoogle Scholar
  33. 33.
    Han, G., Que, W., Jia, G., Shu, L.: An efficient virtual machine consolidation scheme for multimedia cloud computing. Sensors 16(2), 246–262 (2016)CrossRefGoogle Scholar
  34. 34.
    Cho, K.M., Tsai, P.W., Tsai, C.W., Yang, C.S.: A hybrid meta-heuristic algorithm for VM scheduling with load balancing in cloud computing. Neural Comput. Appl. 26(6), 1–13 (2014)Google Scholar
  35. 35.
    Chowdhury, M.R., Mahmud, M.R., Rahman, R.M.: Implementation and performance analysis of various VM placement strategies in CloudSim. J. Cloud Comput. 4(1), 1–21 (2015)CrossRefGoogle Scholar
  36. 36.
    Garg, S.K., Toosi, A.N., Gopalaiyengar, S.K., Buyya, R.: SLA-based virtual machine management for heterogeneous workloads in a cloud datacenter. J. Netw. Comput. Appl. 45(4), 108–120 (2014)CrossRefGoogle Scholar
  37. 37.
    Park, K.S., Pai, V.S.: CoMon: a mostly-scalable monitoring system for PlanetLab. ACM SIGOPS Oper. Syst Rev. 40(1), 65–74 (2006)CrossRefGoogle Scholar
  38. 38.
    Calheiros, R.N., Ranjan, R., Beloglazov, A., Rose, C.A.F.D., Buyya, R.: CloudSim: a toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms. Software 41(1):23–50 (2011)Google Scholar
  39. 39.
    Voorsluys, W., Broberg, J., Venugopal, S., Buyya, R.: Cost of virtual machine live migration in clouds: a performance evaluation. In: Proceeding of IEEE International Conference on Cloud Computing, pp. 254–265 (2009)Google Scholar
  40. 40.
    Liu, K.: Applied Markov Decision Processes, pp. 33–41. Tsinghua University Press, Beijing (2004)Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of Computer Science and Technology, School of Internet of Things EngineeringJiangnan UniversityWuxiChina

Personalised recommendations