Cluster Computing

, Volume 20, Issue 3, pp 2083–2094 | Cite as

A network aware approach for the scheduling of virtual machine migration during peak loads

  • Martin DugganEmail author
  • Jim Duggan
  • Enda Howley
  • Enda Barrett


Live virtual machine migration can have a major impact on how a cloud system performs, as it consumes significant amounts of network resources such as bandwidth. Migration contributes to an increase in consumption of network resources which leads to longer migration times and ultimately has a detrimental effect on the performance of a cloud computing system. Most industrial approaches use ad-hoc manual policies to migrate virtual machines. In this paper, we propose an autonomous network aware live migration strategy that observes the current demand level of a network and performs appropriate actions based on what it is experiencing. The Artificial Intelligence technique known as Reinforcement Learning acts as a decision support system, enabling an agent to learn optimal scheduling times for live migration while analysing current network traffic demand. We demonstrate that an autonomous agent can learn to utilise available resources when peak loads saturate the cloud network.


Live migration Bandwidth Reinforcement learning Network flow Peak load 


  1. 1.
    Akoush, S., Sohan, R., Rice, A., Moore, A.W., Hopper, A.: Predicting the performance of virtual machine migration. In: 2010 IEEE International Symposium on Modeling, Analysis & Simulation of Computer and Telecommunication Systems (MASCOTS), pp. 37–46. IEEE, New York (2010)Google Scholar
  2. 2.
    Amoui, M., Salehie, M., Mirarab, S., Tahvildari, L.: Adaptive action selection in autonomic software using reinforcement learning. In: Fourth International Conference on Autonomic and Autonomous Systems, 2008 (ICAS 2008), pp. 175–181. IEEE, New York (2008)Google Scholar
  3. 3.
    Armbrust, M., Fox, A., Griffith, R., Joseph, A.D., Katz, R., Konwinski, A., Lee, G., Patterson, D., Rabkin, A., Stoica, I., et al.: A view of cloud computing. Commun. ACM 53(4), 50–58 (2010)CrossRefGoogle Scholar
  4. 4.
    Bahati, R.M., Bauer, M., et al.: Towards adaptive policy based management. In: 2010 IEEE Network Operations and Management Symposium (NOMS), pp. 511–518. IEEE, New York (2010)Google Scholar
  5. 5.
    Barrett, E., Howley, E., Duggan, J.: A learning architecture for scheduling workflow applications in the cloud. In: 2011 Ninth IEEE European Conference on Web Services (ECOWS), pp. 83–90. IEEE, New York (2011)Google Scholar
  6. 6.
    Barrett, E., Howley, E., Duggan, J.: Applying reinforcement learning towards automating resource allocation and application scalability in the cloud. Concurr. Comput. Pract. Exp. 25(12), 1656–1674 (2013)CrossRefGoogle Scholar
  7. 7.
    Beloglazov, A., Buyya, R.: Energy efficient resource management in virtualized cloud data centers. In: Proceedings of the 2010 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing. pp. 826– 831. IEEE Computer Society, Washington, DC (2010)Google Scholar
  8. 8.
    Beloglazov, A., Buyya, R.: Optimal online deterministicalgorithms and adaptive heuristics for energy and performance efficient dynamic consolidation of virtual machines in cloud data centers. Concurr. Comput. Pract. Exp. 24(13), 1397–1420 (2012)CrossRefGoogle Scholar
  9. 9.
    Chen, J., Liu, W., Song, J.: Network performanceaware virtual machine migration in data centers. Citeseer (2012)Google Scholar
  10. 10.
    Chen, H., Kang, H., Jiang, G., Zhang, Y.: Coordinating Virtual Machine Migrations in Enterprise Data Centers and Clouds. IEEE, New York (2013)Google Scholar
  11. 11.
    Clark, C., Fraser, K., Hand, S., Hansen, J.G., Jul, E.,Limpach, C., Pratt, I., Warfield, A.: Live migration of virtual machines. In: Proceedings of the 2nd Conference on Symposium on Networked Systems Design & Implementation, vol. 2, pp. 273–286. USENIX Association, San Diego (2005)Google Scholar
  12. 12.
    Duggan, M., Duggan, J., Howley, E., Barrett, E.: A reinforcement learning approach for the scheduling of live migration from under utilised hosts. Memet. Comput. 8, 111 (2016). doi: 10.1007/s12293-016-0218-x Google Scholar
  13. 13.
    Duggan, M., Duggan, J., Howley, E., Barrett, E.: A reinforcement learning approach for dynamic selection of virtual machines in cloud data centres. In: Conference: Innovative Computing Technology (INTECH 2016) (2016)Google Scholar
  14. 14.
    Dutreilh, X., Kirgizov, S., Melekhova, O., Malenfant, J., Rivierre, N., Truck, I.: Using reinforcement learning for autonomic resource allocation in clouds: Towards a fully automated workflow. In: ICAS 2011, The Seventh International Conference on Autonomic and Autonomous Systems. pp. 67–74 (2011)Google Scholar
  15. 15.
    Farahnakian, F., Liljeberg, P., Plosila, J.: Energy-efficient virtual machines consolidation in cloud data centers using reinforcement learning. In: 2014 22nd Euromicro International Conference on Parallel, Distributed and Network-Based Processing (PDP), pp. 500–507. IEEE, New York (2014)Google Scholar
  16. 16.
    Garg, S.K., Buyya, R.: Networkcloudsim: modelling parallel applications in cloud simulations. In: 2011 Fourth IEEE International Conference on Utility and Cloud Computing (UCC), pp. 105–113. IEEE, New York (2011)Google Scholar
  17. 17.
    Ghorbani, S., Caesar, M.: Walk the line: consistent network updates with bandwidth guarantees. In: Proceedings of the First Workshop on Hot Topics in Software Defined Networks, pp. 67–72. ACM, New York (2012)Google Scholar
  18. 18.
    Hu, K., Sim, A., Antoniades, D., Dovrolis, C.: Estimating and forecasting network traffic performance based on statistical patterns observed in SNMP data. In: Machine Learning and Data Mining in Pattern Recognition, pp. 601–615. Springer, New York (2013)Google Scholar
  19. 19.
    Ipek, E., Mutlu, O., Mart’ınez, J.F., Caruana, R.: Selfoptimizing memory controllers: a reinforcement learning approach. In: 35th International Symposium on Computer Architecture, 2008 (ISCA’08), pp. 39–50. IEEE, New York (2008)Google Scholar
  20. 20.
    Mandal, U., Habib, M.F., Zhang, S., Chowdhury, P., Tornatore, M., Mukherjee, B.: Heterogeneous bandwidth provisioning for virtual machine migration over SDN enabled optical networks. In: Optical Fiber Communication Conference. pp. M3H–2. Optical Society of America, Washington, DC (2014)Google Scholar
  21. 21.
    Mandal, U., Habib, M.F., Zhang, S., Tornatore, M., Mukherjee, B.: Bandwidth and routing assignment for virtual machine migration in photonic cloud networks. In: 39th European Conference and Exhibition on Optical Communication (ECOC 2013) (2013)Google Scholar
  22. 22.
    Mason, K., Mannion, P., Duggan, J., Howley, E.: Applying multi-agent reinforcement learning to watershed management. In: Proceedings of the Adaptive and Learning Agents Workshop (2016)Google Scholar
  23. 23.
    Piao, J.T., Yan, J.: A network-aware virtual machine placement and migration approach in cloud computing. In: 2010 9th International Conference on Grid and Cooperative Computing (GCC), pp. 87–92. IEEE, New York (2010)Google Scholar
  24. 24.
    Sajad, M., Patrick, M., Enda, H., Shaukat, M.: Traffic light control using deep policy gradient reinforcement learningGoogle Scholar
  25. 25.
    Stage, A., Setzer, T.: Network-aware migration control and scheduling of differentiated virtual machine workloads. In: Proceedings of the 2009 ICSE Workshop on Software Engineering Challenges of Cloud Computing, pp. 9–14. IEEE Computer Society, Washington, DC (2009)Google Scholar
  26. 26.
    Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction, vol. 1. MIT Press, Cambridge (1998)Google Scholar
  27. 27.
    Tan, Y., Liu, W., Qiu, Q.: Adaptive power management using reinforcement learning. In: Proceedings of the 2009 International Conference on Computer-Aided Design. pp. 461–467. ACM, New York (2009)Google Scholar
  28. 28.
    Tesauro, G., Jong, N.K., Das, R., Bennani, M.N.: Ahybrid reinforcement learning approach to autonomic resource allocation. In: IEEE International Conference on Autonomic Computing, 2006 (ICAC’06), pp. 65–73. IEEE, New York (2006)Google Scholar
  29. 29.
    Verma, A., Ahuja, P., Neogi, A.: pmapper: power andmigration cost aware application placement in virtualized systems. In: Middleware 2008, pp. 243–264. Springer, New York (2008)Google Scholar
  30. 30.
    Watkins, C.J., Dayan, P.: Q-learning. Mach. Learn. 8(3–4), 279–292 (1992)zbMATHGoogle Scholar
  31. 31.
    Wood, T., Ramakrishnan, K., Shenoy, P., Van der Merwe, J., Hwang, J., Liu, G., Chaufournier, L.: Cloudnet: dynamic pooling of cloud resources by live wan migration of virtual machines. IEEE/ACM Trans. Netw. (TON) 23(5), 1568–1583 (2015)CrossRefGoogle Scholar
  32. 32.
    Wood, T., Shenoy, P.J., Venkataramani, A., Yousif, M.S.: Black-box and gray-box strategies for virtual machine migration. NSDI 7, 17–17 (2007)Google Scholar
  33. 33.
    Yuan, J., Miao, X., Li, L., Jiang, X.: An online energysaving resource optimization methodology for data center. J. Softw. 8(8), 1875–1880 (2013)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2017

Authors and Affiliations

  • Martin Duggan
    • 1
    Email author
  • Jim Duggan
    • 1
  • Enda Howley
    • 1
  • Enda Barrett
    • 1
  1. 1.College of Engineering and InformaticsNational University of IrelandGalwayIreland

Personalised recommendations