Abstract
Physical machines in modern datacenters are routinely upgraded due to their maintenance requirements, which involves migrating all the virtual machines they currently host to alternative physical machines. For this kind of datacenter upgrades, it is critical to minimize the time it takes to upgrade all the physical machines in the datacenter, so as to reduce disruptions to cloud services. To minimize the upgrade time, it is essential to carefully schedule the migration of virtual machines on each physical machine during its upgrade, without violating any constraints imposed by virtual machines that are currently running. Rather than resorting to heuristic algorithms, we propose a new scheduler, Raven, that uses an experience-driven approach with deep reinforcement learning to schedule the virtual machine migration process. With our design of the state space, action space and reward function, Raven trains a fully-connected neural network using the cross-entropy method to approximate the policy of a choosing destination physical machine for each migrating virtual machine. We compare Raven with state-of-the-art heuristic algorithms in the literature, and our results show that Raven effectively leads to shorter time to complete the datacenter upgrade process.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Maurya, K., Sinha, R.: Energy conscious dynamic provisioning of virtual machines using adaptive migration thresholds in cloud data center. Int. J. Comput. Sci. Mob. Comput. 2(3), 74–82 (2013)
Ji, S., Li, M.D., Ji, N., Li, B.: An online virtual machine placement algorithm in an over-committed cloud. In: 2018 IEEE International Conference on Cloud Engineering, IC2E 2018, Orlando, FL, USA, 17–20 April 2018, pp. 106–112 (2018)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction, vol. 1, no. 1. MIT Press, Cambridge (1998)
Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518, 529–533 (2015)
Mao, H., Alizadeh, M., Menache, I., Kandula, S.: Resource management with deep reinforcement learning. In: Proceedings of the 15th ACM Workshop on Hot Topics in Networks, pp. 50–56. ACM (2016)
Silver, D., et al.: Mastering the game of go without human knowledge. Nature 550, 354–359 (2017)
Rubinstein, R., Kroese, D.: The Cross-Entropy Method. Springer, Heidelberg (2004). https://doi.org/10.1007/978-1-4757-4321-0
Sapuntzakis, C.P., Chandra, R., Pfaff, B., Chow, J., Lam, M.S., Rosenblum, M.: Optimizing the migration of virtual computers. In: 5th Symposium on Operating System Design and Implementation (OSDI 2002), Boston, Massachusetts, USA, 9–11 December 2002 (2002)
Zhang, X., Shae, Z.-Y., Zheng, S., Jamjoom, H.: Virtual machine migration in an over-committed cloud. In: Proceedings of the IEEE Network Operations and Management Symposium (NOMS) (2012)
Dabbagh, M., Hamdaoui, B., Guizani, M., Rayes, A.: Efficient datacenter resource utilization through cloud resource overcommitment. In: Proceedings of the IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS) (2015)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Ying, C., Li, B., Ke, X., Guo, L. (2020). Scheduling Virtual Machine Migration During Datacenter Upgrades with Reinforcement Learning. In: Chu, X., Jiang, H., Li, B., Wang, D., Wang, W. (eds) Quality, Reliability, Security and Robustness in Heterogeneous Systems. QShine 2019. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 300. Springer, Cham. https://doi.org/10.1007/978-3-030-38819-5_7
Download citation
DOI: https://doi.org/10.1007/978-3-030-38819-5_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-38818-8
Online ISBN: 978-3-030-38819-5
eBook Packages: Computer ScienceComputer Science (R0)