Advertisement

Cluster Computing

, Volume 20, Issue 3, pp 1995–2012 | Cite as

AutoMigrate: a framework for developing intelligent, self-managing cloud services with maximum availability

  • Mamadou H. DialloEmail author
  • Michael August
  • Roger Hallman
  • Megan Kline
  • Scott M. Slayback
  • Christopher Graves
Article

Abstract

Cloud services are on-demand services provided to end-users over the Internet and hosted by cloud service providers. A cloud service consists of a set of interacting applications/processes running on one or more interconnected VMs. Organizations are increasingly using cloud services as a cost-effective means for outsourcing their IT departments. However, cloud service availability is not guaranteed by cloud service providers, especially in the event of anomalous circumstances that spontaneously disrupt availability including natural disasters, power failure, and cybersecurity attacks. In this paper, we propose a framework for developing intelligent systems that can monitor and migrate cloud services to maximize their availability in case of cloud disruption. The framework connects an autonomic computing agent to the cloud to automatically migrate cloud services based on anticipated cloud disruption. The autonomic agent employs a modular design to facilitate the incorporation of different techniques for deciding when to migrate cloud services, what cloud services to migrate, and where to migrate the selected cloud services. We incorporated a virtual machine selection algorithm for deciding what cloud services to migrate that maximizes the availability of high priority services during migration under time and network bandwidth constraints. We implemented the framework and conducted experiments to evaluate the performance of the underlying techniques. Based on the experiments, the use of this framework results in less down-time due to migration, thereby leading to reduced cloud service disruption.

Keywords

Cloud computing Anomaly detection Self-managing cloud services Intelligent systems Cloud service availability Live migration of virtual machines 

Notes

Acknowledgements

We would like to thank Luis Angel Bathen for his contributions in starting this project. We also would like to thank Vic Beach, Tonya R. Nishio, and Ronald A. Wolfe for their managerial support.

References

  1. 1.
    Shrivastava, V., Zerfos, P., Lee, K., Jamjoom, H., Liu, Y., Banerjee, S.: Application-aware virtual machine migration in data centers. In: INFOCOM 2011. 30th IEEE International Conference on Computer Communications, Joint Conference of the IEEE Computer and Communications Societies, 10–15 April 2011, Shanghai, China, pp. 66–70 (2011)Google Scholar
  2. 2.
    Clark, C., Fraser, K., Hand, S., Hansen, J.G., Jul, E., Limpach, C., Pratt, I., Warfield, A.: Live migration of virtual machines. In: Proceedings of the 2nd Conference on Symposium on Networked Systems Design & Implementation (NSDI’05), Vol. 2, pp. 273–286. USENIX Association, Berkeley, CA, USA (2005)Google Scholar
  3. 3.
    Huang, T., Zhu, Y., Wu, Y., Bressan, S., Dobbie, G.: Anomaly detection and identification scheme for VM live migration in cloud infrastructure. Futur. Gener. Comput. Syst. 56, 736–745 (2016)CrossRefGoogle Scholar
  4. 4.
    Nagafuchi, Y., Teramoto, Y., Hu, B., Kishi, T., Koyama, T., Kitazume, H.: Routing optimization for live VM migration between datacenters. In: 2015 10th Asia-Pacific Symposium on Information and Telecommunication Technologies (APSITT), pp. 1–3. IEEE (2015)Google Scholar
  5. 5.
    Shetty, S., Yuchi, X., Song, M.: Towards a network-aware VM migration: Evaluating the cost of vm migration in cloud data centers. In: Moving Target Defense for Distributed Systems. Springer, Berlin (2016)Google Scholar
  6. 6.
    Li, X., He, Q., Chen, J., Ye, K., Yin, T.: Informed live migration strategies of virtual machines for cluster load balancing. IEEE Trans. Softw. Eng. PP(99), 111–122 (2011)Google Scholar
  7. 7.
    Lu, P., Barbalace, A., Palmieri, R., Ravindran, B.: Adaptive live migration to improve load balancing in virtual machine environment. In: Euro-Par 2013: Parallel Processing Workshops 2013, Aachen, Germany, August 26–27, 2013. Revised Selected Papers, pp. 116–125 (2013)Google Scholar
  8. 8.
    Wood, T., Shenoy, P.J., Venkataramani, A., Yousif, M.S.: Black-box and gray-box strategies for virtual machine migration. NSDI 7, 17–17 (2007)Google Scholar
  9. 9.
    Zhao, Y., Huang, W.: Adaptive distributed load balancing algorithm based on live migration of virtual machines in cloud. In: International Conference on Networked Computing and Advanced Information Management, NCM 2009, pp. 170–175 (2009)Google Scholar
  10. 10.
    Beloglazov, A., Abawajy, J.H., Buyya, R.: Energy-aware resource allocation heuristics for efficient management of data centers for cloud computing. Futur. Gener. Comp. Syst. 28(5), 755–768 (2012)CrossRefGoogle Scholar
  11. 11.
    Beloglazov, A., Buyya, R.: Energy efficient resource management in virtualized cloud data centers. In: Proceedings of the 2010 10th IEEE/ACM international conference on cluster, cloud and grid computing, pp. 826–831. IEEE Computer Society (2010)Google Scholar
  12. 12.
    Lin, M., Wierman, A., Andrew, L.L.H., Thereska, E.: Dynamic right-sizing for power-proportional data centers. IEEE/ACM Trans. Netw. 21(5), 1378–1391 (2013)CrossRefGoogle Scholar
  13. 13.
    Liu, H., Jin, H., Xu, C., Liao, X.: Performance and energy modeling for live migration of virtual machines. Clust. Comput. 16(2), 249–264 (2013)CrossRefGoogle Scholar
  14. 14.
    Strunk, A., Dargie, W.: Does live migration of virtual machines cost energy? In: 2013 IEEE 27th International Conference on advanced Information Networking and Applications (AINA), pp. 514–521 (2013)Google Scholar
  15. 15.
    Calcavecchia, N.M., Biran, O., Hadad, E., Moatti, Y.: Vm placement strategies for cloud scenarios. In: Cloud Computing (CLOUD), 2012 IEEE 5th International Conference on, pp. 852–859. IEEE (2012)Google Scholar
  16. 16.
    Hines, M.R., Deshpande, U., Gopalan, K.: Post-copy live migration of virtual machines. Oper. Syst. Rev. 43(3), 14–26 (2009)CrossRefGoogle Scholar
  17. 17.
    Strunk, A.: Costs of virtual machine live migration: a survey. In: Eighth IEEE World Congress on Services, SERVICES 2012, Honolulu, June 24–29, 2012, pp. 323–329 (2012)Google Scholar
  18. 18.
    Voorsluys, W., Broberg, J., Venugopal, S., Buyya, R.: Cost of virtual machine live migration in clouds: a performance evaluation. CoRR (2011). arXiv:1109.4974
  19. 19.
    pykalman: the kalman filter, kalman smoother, and em library for python (2017). http://pykalman.github.io
  20. 20.
    Masdari, M., Nabavi, S.S., Ahmadi, V.: An overview of virtual machine placement schemes in cloud computing. J. Netw. Comput. Appl. 66, 106–127 (2016). doi: 10.1016/j.jnca.2016.01.011
  21. 21.
    Ahmad, R.W., Gani, A., Hamid, S.H.A., Shiraz, M., Xia, F., Madani, S.A.: Virtual machine migration in cloud data centers: a review, taxonomy, and open research issues. J. Supercomput. 71(7), 2473–2515 (2015)CrossRefGoogle Scholar
  22. 22.
    Arulselvan, A.: A note on the set union knapsack problem. Discret. Appl. Math. 169, 214–218 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  23. 23.
    Goldschmidt, O., Nehme, D., Yu, G.: Note: on the set-union knapsack problem. Nav. Res. Logist. (NRL) 41(6), 833 (1994)CrossRefzbMATHGoogle Scholar
  24. 24.
    Project, X.: Xen project: a linux foundation collaborative project. (2016). http://www.xenproject.org/
  25. 25.
    The apache thrift software framework (2016). https://thrift.apache.org/
  26. 26.
    scikit-learn: Machine learning in python (2016). http://scikit-learn.org/stable/
  27. 27.
    Vedhanayagam, P., S., S., Balusamy, B., Vijayakumar, P., Chang, V.: Analysis of measures to achieve resilience during virtual machine interruptions in iaas cloud service. In: Proceedings of the International Conference on Internet of Things, Big Data and Security, Vol. 1, IoTBDS (2017)Google Scholar
  28. 28.
    Barbhuiya, S., Papazachos, Z., Kilpatrick, P., Nikolopoulos, D.S.: A lightweight tool for anomaly detection in cloud data centres. In: Proceedings of the 5th International Conference on Cloud Computing and Services Science - Volume 1: CLOSER,, pp. 343–351 (2015). doi: 10.5220/0005453403430351
  29. 29.
    Mhedheb, Y., Streit, A.: Energy-efficient task scheduling in data centers. In: CLOSER 2016—Proceedings of the 6th International Conference on Cloud Computing and Services Science, Vol. 1, April 23–25, pp. 273–282, Rome (2016). doi: 10.5220/0005880802730282
  30. 30.
    Lou, J.G., Fu, Q., Yang, S., Xu, Y., Li, J.: Mining invariants from console logs for system problem detection. In: USENIX Annual Technical Conference (2010)Google Scholar
  31. 31.
    Xu, W., Huang, L., Fox, A., Patterson, D., Jordan, M.I.: Detecting large-scale system problems by mining console logs. In: Proceedings of the ACM SIGOPS 22nd Symposium on Operating Systems Principles, pp. 117–132. ACM (2009)Google Scholar
  32. 32.
    Wang, C.: Ebat: online methods for detecting utility cloud anomalies. In: Proceedings of the 6th Middleware Doctoral Symposium, p. 4. ACM (2009)Google Scholar
  33. 33.
    Kang, H., Chen, H., Jiang, G.: Peerwatch: a fault detection and diagnosis tool for virtualized consolidation systems. In: Proceedings of the 7th international conference on Autonomic computing, pp. 119–128. ACM (2010)Google Scholar
  34. 34.
    Ye, K., Jiang, X., Huang, D., Chen, J., Wang, B.: Live migration of multiple virtual machines with resource reservation in cloud computing environments. In: IEEE International Conference on Cloud Computing, CLOUD, Washington, DC, pp. 267–274 (2011)Google Scholar
  35. 35.
    Deshpande, U., Wang, X., Gopalan, K.: Live gang migration of virtual machines. In: Proceedings of the 20th International Symposium on High Performance Distributed Computing, pp. 135–146. ACM (2011)Google Scholar
  36. 36.
    Sun, G., Liao, D., Anand, V., Zhao, D., Yu, H.: A new technique for efficient live migration of multiple virtual machines. Futur. Gener. Comput. Syst. 55, 74–86 (2016)CrossRefGoogle Scholar
  37. 37.
    Song, X., Shi, J., Liu, R., Yang, J., Chen, H.: Parallelizing live migration of virtual machines. In: ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments, Houston, TX, USA, March 16–17, pp. 85–96 (2013)Google Scholar
  38. 38.
    Liu, H., He, B.: Vmbuddies: coordinating live migration of multi-tier applications in cloud environments. IEEE Trans. Parallel Distrib. Syst. 26(4), 1192–1205 (2015)CrossRefGoogle Scholar
  39. 39.
    Berthier, N., Rutten, E., Depalma, N., Gueye, S.: Designing autonomic management systems by using reactive control techniques. IEEE Trans. Softw. Eng. (99), 1–1 (2015)Google Scholar
  40. 40.
    Deshpande, U., Keahey, K.: Traffic-sensitive live migration of virtual machines. In: 15th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGrid 2015, Shenzhen, China, May 4-7, 2015, pp. 51–60 (2015)Google Scholar
  41. 41.
    Rybina, K., Patni, A., Schill, A.: Analysing the migration time of live migration of multiple virtual machines. In: Proceedings of the 4th International Conference on Cloud Computing and Services Science, pp. 590–597 (2014)Google Scholar
  42. 42.
    Dong, D., Herbert, J.: Precise VM placement algorithm supported by data analytic service. In: CLOSER, pp. 463–468 (2013)Google Scholar
  43. 43.
    Fang, W., Liang, X., Li, S., Chiaraviglio, L., Xiong, N.: Vmplanner: optimizing virtual machine placement and traffic flow routing to reduce network power costs in cloud data centers. Comput. Netw. 57(1), 179–196 (2013)CrossRefGoogle Scholar
  44. 44.
    Diallo, M.H., August, M., Hallman, R., Kline, M., Au, H., Beach, V.: Nomad: A framework for developing mission-critical cloud-based applications. In: 10th International Conference on Availability, Reliability and Security, ARES 2015, Toulouse, France, August 24–27, 2015, pp. 660–669 (2015)Google Scholar
  45. 45.
    Diallo, M.H., August, M.A., Hallman, R.A., Kline, M., Au, H., Beach, V.: Callforfire: A mission-critical cloud-based application built using the nomad framework. In: Twentieth International Conference on Financial Cryptography and Data Security: 4th Workshop on Encrypted Computing and Applied Homomorphic Cryptography, Christ Church, Barbados, February 22–26 (2016)Google Scholar

Copyright information

© Springer Science+Business Media New York (outside the USA) 2017

Authors and Affiliations

  • Mamadou H. Diallo
    • 1
    Email author
  • Michael August
    • 1
  • Roger Hallman
    • 1
  • Megan Kline
    • 1
  • Scott M. Slayback
    • 1
  • Christopher Graves
    • 1
  1. 1.US Department of Defense SPAWAR Systems Center Pacific (SSC Pacific)San DiegoUSA

Personalised recommendations