Skip to main content

A reinforcement learning approach for the scheduling of live migration from under utilised hosts

Memetic Computing volume 9, pages 283–293 (2017)Cite this article

Abstract

Live virtual machine migration can have a major impact on how a cloud system performs, as it consumes significant amount of network resources, such as bandwidth. A virtual machine migration occurs when a host becomes over-utilised or under-utilised. In this paper, we propose a network aware live migration strategy that monitors the current demand level of bandwidth when network congestion occurs and performs appropriate actions based on what it is experiencing. The Artificial Intelligence technique that is based on Reinforcement Learning acts as a decision support system, enabling an agent to learn an optimal time to schedule a virtual machine migration depending on the current bandwidth usage in a data centre. We show from our results that an autonomous agent can learn to utilise available network resources such as bandwidth when network saturation occurs at peak times.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price includes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Notes

  1. Environmental Protection Agency (EPA) website link http://www.epa.ie/climate/calculators/.VlWsEnbhBQI.

References

  1. Akoush S, Sohan R, Rice A, Moore AW, Hopper A (2010) Predicting the performance of virtual machine migration. In: 2010 IEEE international symposium on modeling, analysis and simulation of computer and telecommunication systems. IEEE, pp 37–46

  2. Armbrust M, Fox A, Griffith R, Joseph AD, Katz R, Konwinski A, Lee G, Patterson D, Rabkin A, Stoica I et al (2010) A view of cloud computing. Commun ACM 53(4):50–58

    Article  Google Scholar 

  3. Bahati RM, Bauer MA (2010) Towards adaptive policy-based management. In: 2010 IEEE network operations and management symposium-NOMS. IEEE, pp 511–518

  4. Barrett E, Howley E, Duggan J (2013) Applying reinforcement learning towards automating resource allocation and application scalability in the cloud. Concurr Comput 25(12):1656–1674

    Article  Google Scholar 

  5. Barroso LA, Hólzle U (2007) The case for energy proportional computing. Computer 40(12):33–37. doi:10.1109/MC.2007.443

  6. Beloglazov A, Buyya R (2012) Optimal online deterministic algorithms and adaptive heuristics for energy and performance efficient dynamic consolidation of virtual machines in cloud data centers. Concurr Comput Pract Exp 24(13):13197–1420

    Article  Google Scholar 

  7. Beloglazov A, Buyya R, Lee YC, Zomaya A (2011) A taxonomy and survey of energy-efficient data centers and cloud computing systems, vol 82. Academic Press, USA

    Google Scholar 

  8. Chen J, Liu W, Song J (2012) Network performance aware virtual machine migration in data centers. In: CLOUD COMPUTING 2012: the third international conference on cloud computing, GRIDs, and virtualization, pp 65–71

  9. Chen H, Kang H, Jiang G, Zhang Y (2013) Network-aware coordination of virtual machine migrations in enterprise data centers and clouds. In: 2013 IFIP/IEEE international symposium on integrated network management (IM 2013). IEEE, pp 888–891

  10. Clark C, Fraser K, Hand S, Hansen JG, Jul E, Limpach C, Pratt I, Warfield A (2005) Live migration of virtual machines. Proceedings of the 2nd conference on symposium on networked systems design & implementation, vol 2. USENIX Association, Berkeley, pp 273–286

    Google Scholar 

  11. Duggan M, Flesk K, Duggan J, Howley E, Barrett E (2016) A reinforcement learning approach for dynamic selection of virtual machines in cloud data centres. In: Sixth International Conference on Innovating Computing Technology. IEEE

  12. Dutreilh X, Kirgizov S, Melekhova O, Malenfant J, Rivierre N, Truck I (2011) Using reinforcement learning for autonomic resource allocation in clouds: towards a fully automated workflow. In: ICAS 2011, the seventh international conference on autonomic and autonomous systems, pp 67–74

  13. Farahnakian F, Liljeberg P, Plosila J (2014) Energy-efficient virtual machines consolidation in cloud data centers using reinforcement learning. In: 2014 22nd Euromicro international conference on parallel, distributed, and network based processing. IEEE, pp 500–507

  14. Ghorbani S, Caesar M (2012) Walk the line: consistent network updates with bandwidth guarantees. Proceedings of the first workshop on Hot topics in software defined networks. ACM, New York, pp 67–72

    Chapter  Google Scholar 

  15. Hu K, Sim A, Antoniades D, Dovrolis C (2013) Estimating and forecasting network traffic performance based on statistical patterns observed in snmp data. International workshop on machine learning and data mining in pattern recognition. Springer, Berlin, pp 601–615

  16. Koomey J (2011) Growth in data center electricity use 2005 to 2010. A report by Analytical Press, completed at the request of The New York Times, vol 9

  17. Mandal U, Habib MF, Zhang S, Chowdhury P, Tornatore M, Mukherjee B (2014) Heterogeneous bandwidth provisioning for virtual machine migration over sdn-enabled optical networks. Optical fiber communication conference. Optical Society of America, USA, pp M3H–2

    Google Scholar 

  18. Mandal U, Habib MF, Zhang S, Tornatore M, Mukherjee B (2013) Bandwidth and routing assignment for virtual machine migration in photonic cloud networks. In: IET conference proceedings. The Institution of Engineering & Technology

  19. Piao JT, Yan J (2010) A network-aware virtual machine placement and migration approach in cloud computing. 2010 ninth international conference on grid and cloud computing. IEEE Computer Society, Washington, pp 87–92

    Chapter  Google Scholar 

  20. Stage A, Setzer T (2009) Network-aware migration control and scheduling of differentiated virtual machine workloads. Proceedings of the 2009 ICSE workshop on software engineering challenges of cloud computing. IEEE Computer Society, Washington, pp 9–14

    Chapter  Google Scholar 

  21. Sutton RS, Barto AG (1998) Reinforcement learning: an introduction, vol 1. MIT press, Cambridge

    Google Scholar 

  22. Tan Y, Liu W, Qiu Q (2009) Adaptive power management using reinforcement learning. Proceedings of the 2009 international conference on computer-aided design. ACM, USA, pp 461–467

    Google Scholar 

  23. Tesauro G, Jong NK, Das R, Bennani MN (2006) A hybrid reinforcement learning approach to autonomic resource allocation. In: 2006 IEEE international conference on autonomic computing. IEEE, pp 65–73

  24. Verma A, Ahuja P, Neogi A (2008) pMapper: power and migration cost aware application placement in virtualized systems. ACM/IFIP/USENIX international conference on distributed systems platforms and open distributed processing. Springer, New York, pp 243–264

    Google Scholar 

  25. Watkins CJ, Dayan P (1992) Q-learning. Mach Learn 8(3–4):279–292

    MATH  Google Scholar 

  26. Wood T, Ramakrishnan K, Shenoy P, Van der Merwe J, Hwang J, Liu G, Chaufournier L (2015) Cloudnet: dynamic pooling of cloud resources by live wan migration of virtual machines. IEEE/ACM Trans Netw (TON) 23(5):1568–1583

    Article  Google Scholar 

  27. Wood T, Shenoy PJ, Venkataramani A, Yousif MS (2007) Black-box and gray-box strategies for virtual machine migration. NSDI 7:17–17

    Google Scholar 

  28. Yuan J, Miao X, Li L, Jiang X (2013) An online energy saving resource optimization methodology for data center. J Softw 8(8):1875–1880

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. College of Engineering and Informatics National University of Ireland, University Road, Galway, Ireland

    Martin Duggan, Jim Duggan, Enda Howley & Enda Barrett

Authors
  1. Martin Duggan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jim Duggan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Enda Howley
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Enda Barrett
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Martin Duggan.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Duggan, M., Duggan, J., Howley, E. et al. A reinforcement learning approach for the scheduling of live migration from under utilised hosts. Memetic Comp. 9, 283–293 (2017). https://doi.org/10.1007/s12293-016-0218-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12293-016-0218-x

Keywords

  • Reinforcement
  • Learning
  • Live
  • Migration
  • Bandwidth