Skip to main content
SpringerLink
Log in

AutoMigrate: a framework for developing intelligent, self-managing cloud services with maximum availability

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

Cloud services are on-demand services provided to end-users over the Internet and hosted by cloud service providers. A cloud service consists of a set of interacting applications/processes running on one or more interconnected VMs. Organizations are increasingly using cloud services as a cost-effective means for outsourcing their IT departments. However, cloud service availability is not guaranteed by cloud service providers, especially in the event of anomalous circumstances that spontaneously disrupt availability including natural disasters, power failure, and cybersecurity attacks. In this paper, we propose a framework for developing intelligent systems that can monitor and migrate cloud services to maximize their availability in case of cloud disruption. The framework connects an autonomic computing agent to the cloud to automatically migrate cloud services based on anticipated cloud disruption. The autonomic agent employs a modular design to facilitate the incorporation of different techniques for deciding when to migrate cloud services, what cloud services to migrate, and where to migrate the selected cloud services. We incorporated a virtual machine selection algorithm for deciding what cloud services to migrate that maximizes the availability of high priority services during migration under time and network bandwidth constraints. We implemented the framework and conducted experiments to evaluate the performance of the underlying techniques. Based on the experiments, the use of this framework results in less down-time due to migration, thereby leading to reduced cloud service disruption.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Shrivastava, V., Zerfos, P., Lee, K., Jamjoom, H., Liu, Y., Banerjee, S.: Application-aware virtual machine migration in data centers. In: INFOCOM 2011. 30th IEEE International Conference on Computer Communications, Joint Conference of the IEEE Computer and Communications Societies, 10–15 April 2011, Shanghai, China, pp. 66–70 (2011)

  2. Clark, C., Fraser, K., Hand, S., Hansen, J.G., Jul, E., Limpach, C., Pratt, I., Warfield, A.: Live migration of virtual machines. In: Proceedings of the 2nd Conference on Symposium on Networked Systems Design & Implementation (NSDI’05), Vol. 2, pp. 273–286. USENIX Association, Berkeley, CA, USA (2005)

  3. Huang, T., Zhu, Y., Wu, Y., Bressan, S., Dobbie, G.: Anomaly detection and identification scheme for VM live migration in cloud infrastructure. Futur. Gener. Comput. Syst. 56, 736–745 (2016)

    Article  Google Scholar 

  4. Nagafuchi, Y., Teramoto, Y., Hu, B., Kishi, T., Koyama, T., Kitazume, H.: Routing optimization for live VM migration between datacenters. In: 2015 10th Asia-Pacific Symposium on Information and Telecommunication Technologies (APSITT), pp. 1–3. IEEE (2015)

  5. Shetty, S., Yuchi, X., Song, M.: Towards a network-aware VM migration: Evaluating the cost of vm migration in cloud data centers. In: Moving Target Defense for Distributed Systems. Springer, Berlin (2016)

  6. Li, X., He, Q., Chen, J., Ye, K., Yin, T.: Informed live migration strategies of virtual machines for cluster load balancing. IEEE Trans. Softw. Eng. PP(99), 111–122 (2011)

  7. Lu, P., Barbalace, A., Palmieri, R., Ravindran, B.: Adaptive live migration to improve load balancing in virtual machine environment. In: Euro-Par 2013: Parallel Processing Workshops 2013, Aachen, Germany, August 26–27, 2013. Revised Selected Papers, pp. 116–125 (2013)

  8. Wood, T., Shenoy, P.J., Venkataramani, A., Yousif, M.S.: Black-box and gray-box strategies for virtual machine migration. NSDI 7, 17–17 (2007)

    Google Scholar 

  9. Zhao, Y., Huang, W.: Adaptive distributed load balancing algorithm based on live migration of virtual machines in cloud. In: International Conference on Networked Computing and Advanced Information Management, NCM 2009, pp. 170–175 (2009)

  10. Beloglazov, A., Abawajy, J.H., Buyya, R.: Energy-aware resource allocation heuristics for efficient management of data centers for cloud computing. Futur. Gener. Comp. Syst. 28(5), 755–768 (2012)

    Article  Google Scholar 

  11. Beloglazov, A., Buyya, R.: Energy efficient resource management in virtualized cloud data centers. In: Proceedings of the 2010 10th IEEE/ACM international conference on cluster, cloud and grid computing, pp. 826–831. IEEE Computer Society (2010)

  12. Lin, M., Wierman, A., Andrew, L.L.H., Thereska, E.: Dynamic right-sizing for power-proportional data centers. IEEE/ACM Trans. Netw. 21(5), 1378–1391 (2013)

    Article  Google Scholar 

  13. Liu, H., Jin, H., Xu, C., Liao, X.: Performance and energy modeling for live migration of virtual machines. Clust. Comput. 16(2), 249–264 (2013)

    Article  Google Scholar 

  14. Strunk, A., Dargie, W.: Does live migration of virtual machines cost energy? In: 2013 IEEE 27th International Conference on advanced Information Networking and Applications (AINA), pp. 514–521 (2013)

  15. Calcavecchia, N.M., Biran, O., Hadad, E., Moatti, Y.: Vm placement strategies for cloud scenarios. In: Cloud Computing (CLOUD), 2012 IEEE 5th International Conference on, pp. 852–859. IEEE (2012)

  16. Hines, M.R., Deshpande, U., Gopalan, K.: Post-copy live migration of virtual machines. Oper. Syst. Rev. 43(3), 14–26 (2009)

    Article  Google Scholar 

  17. Strunk, A.: Costs of virtual machine live migration: a survey. In: Eighth IEEE World Congress on Services, SERVICES 2012, Honolulu, June 24–29, 2012, pp. 323–329 (2012)

  18. Voorsluys, W., Broberg, J., Venugopal, S., Buyya, R.: Cost of virtual machine live migration in clouds: a performance evaluation. CoRR (2011). arXiv:1109.4974

  19. pykalman: the kalman filter, kalman smoother, and em library for python (2017). http://pykalman.github.io

  20. Masdari, M., Nabavi, S.S., Ahmadi, V.: An overview of virtual machine placement schemes in cloud computing. J. Netw. Comput. Appl. 66, 106–127 (2016). doi:10.1016/j.jnca.2016.01.011

  21. Ahmad, R.W., Gani, A., Hamid, S.H.A., Shiraz, M., Xia, F., Madani, S.A.: Virtual machine migration in cloud data centers: a review, taxonomy, and open research issues. J. Supercomput. 71(7), 2473–2515 (2015)

    Article  Google Scholar 

  22. Arulselvan, A.: A note on the set union knapsack problem. Discret. Appl. Math. 169, 214–218 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  23. Goldschmidt, O., Nehme, D., Yu, G.: Note: on the set-union knapsack problem. Nav. Res. Logist. (NRL) 41(6), 833 (1994)

    Article  MATH  Google Scholar 

  24. Project, X.: Xen project: a linux foundation collaborative project. (2016). http://www.xenproject.org/

  25. The apache thrift software framework (2016). https://thrift.apache.org/

  26. scikit-learn: Machine learning in python (2016). http://scikit-learn.org/stable/

  27. Vedhanayagam, P., S., S., Balusamy, B., Vijayakumar, P., Chang, V.: Analysis of measures to achieve resilience during virtual machine interruptions in iaas cloud service. In: Proceedings of the International Conference on Internet of Things, Big Data and Security, Vol. 1, IoTBDS (2017)

  28. Barbhuiya, S., Papazachos, Z., Kilpatrick, P., Nikolopoulos, D.S.: A lightweight tool for anomaly detection in cloud data centres. In: Proceedings of the 5th International Conference on Cloud Computing and Services Science - Volume 1: CLOSER,, pp. 343–351 (2015). doi:10.5220/0005453403430351

  29. Mhedheb, Y., Streit, A.: Energy-efficient task scheduling in data centers. In: CLOSER 2016—Proceedings of the 6th International Conference on Cloud Computing and Services Science, Vol. 1, April 23–25, pp. 273–282, Rome (2016). doi:10.5220/0005880802730282

  30. Lou, J.G., Fu, Q., Yang, S., Xu, Y., Li, J.: Mining invariants from console logs for system problem detection. In: USENIX Annual Technical Conference (2010)

  31. Xu, W., Huang, L., Fox, A., Patterson, D., Jordan, M.I.: Detecting large-scale system problems by mining console logs. In: Proceedings of the ACM SIGOPS 22nd Symposium on Operating Systems Principles, pp. 117–132. ACM (2009)

  32. Wang, C.: Ebat: online methods for detecting utility cloud anomalies. In: Proceedings of the 6th Middleware Doctoral Symposium, p. 4. ACM (2009)

  33. Kang, H., Chen, H., Jiang, G.: Peerwatch: a fault detection and diagnosis tool for virtualized consolidation systems. In: Proceedings of the 7th international conference on Autonomic computing, pp. 119–128. ACM (2010)

  34. Ye, K., Jiang, X., Huang, D., Chen, J., Wang, B.: Live migration of multiple virtual machines with resource reservation in cloud computing environments. In: IEEE International Conference on Cloud Computing, CLOUD, Washington, DC, pp. 267–274 (2011)

  35. Deshpande, U., Wang, X., Gopalan, K.: Live gang migration of virtual machines. In: Proceedings of the 20th International Symposium on High Performance Distributed Computing, pp. 135–146. ACM (2011)

  36. Sun, G., Liao, D., Anand, V., Zhao, D., Yu, H.: A new technique for efficient live migration of multiple virtual machines. Futur. Gener. Comput. Syst. 55, 74–86 (2016)

    Article  Google Scholar 

  37. Song, X., Shi, J., Liu, R., Yang, J., Chen, H.: Parallelizing live migration of virtual machines. In: ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments, Houston, TX, USA, March 16–17, pp. 85–96 (2013)

  38. Liu, H., He, B.: Vmbuddies: coordinating live migration of multi-tier applications in cloud environments. IEEE Trans. Parallel Distrib. Syst. 26(4), 1192–1205 (2015)

    Article  Google Scholar 

  39. Berthier, N., Rutten, E., Depalma, N., Gueye, S.: Designing autonomic management systems by using reactive control techniques. IEEE Trans. Softw. Eng. (99), 1–1 (2015)

  40. Deshpande, U., Keahey, K.: Traffic-sensitive live migration of virtual machines. In: 15th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGrid 2015, Shenzhen, China, May 4-7, 2015, pp. 51–60 (2015)

  41. Rybina, K., Patni, A., Schill, A.: Analysing the migration time of live migration of multiple virtual machines. In: Proceedings of the 4th International Conference on Cloud Computing and Services Science, pp. 590–597 (2014)

  42. Dong, D., Herbert, J.: Precise VM placement algorithm supported by data analytic service. In: CLOSER, pp. 463–468 (2013)

  43. Fang, W., Liang, X., Li, S., Chiaraviglio, L., Xiong, N.: Vmplanner: optimizing virtual machine placement and traffic flow routing to reduce network power costs in cloud data centers. Comput. Netw. 57(1), 179–196 (2013)

    Article  Google Scholar 

  44. Diallo, M.H., August, M., Hallman, R., Kline, M., Au, H., Beach, V.: Nomad: A framework for developing mission-critical cloud-based applications. In: 10th International Conference on Availability, Reliability and Security, ARES 2015, Toulouse, France, August 24–27, 2015, pp. 660–669 (2015)

  45. Diallo, M.H., August, M.A., Hallman, R.A., Kline, M., Au, H., Beach, V.: Callforfire: A mission-critical cloud-based application built using the nomad framework. In: Twentieth International Conference on Financial Cryptography and Data Security: 4th Workshop on Encrypted Computing and Applied Homomorphic Cryptography, Christ Church, Barbados, February 22–26 (2016)

Download references

Acknowledgements

We would like to thank Luis Angel Bathen for his contributions in starting this project. We also would like to thank Vic Beach, Tonya R. Nishio, and Ronald A. Wolfe for their managerial support.

Author information

Authors and Affiliations

  1. US Department of Defense SPAWAR Systems Center Pacific (SSC Pacific), San Diego, CA, USA

    Mamadou H. Diallo, Michael August, Roger Hallman, Megan Kline, Scott M. Slayback & Christopher Graves

Authors
  1. Mamadou H. Diallo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michael August
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Roger Hallman
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Megan Kline
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Scott M. Slayback
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Christopher Graves
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mamadou H. Diallo.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Diallo, M.H., August, M., Hallman, R. et al. AutoMigrate: a framework for developing intelligent, self-managing cloud services with maximum availability. Cluster Comput 20, 1995–2012 (2017). https://doi.org/10.1007/s10586-017-0900-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10586-017-0900-x

Keywords

Search

Navigation