Skip to main content
SpringerLink
Log in

Agent-based load balancing in Cloud data centers

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

Cloud data centers are generally composed of heterogeneous commodity servers hosting multiple virtual machines (VMs) with potentially different specifications and fluctuating resource usages. This may cause a resource usage imbalance within servers that may result in performance degradation and violations to service level agreements. This work proposes a collaborative agent-based problem solving technique capable of balancing workloads across commodity, heterogeneous servers by making use of VM live migration. The agents are endowed with (i) migration heuristics to determine which VMs should be migrated and their destination hosts, (ii) migration policies to decide when VMs should be migrated, (iii) VM acceptance policies to determine which VMs should be hosted, and (iv) front-end load balancing heuristics. The results show that agents, through autonomous and dynamic collaboration, can efficiently balance loads in a distributed manner outperforming centralized approaches with a performance comparable to commercial solutions, namely Red Hat, while migrating fewer VMs.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Amazon EC2: Elastic load balancing. http://aws.amazon.com/elasticloadbalancing/ (2015). Accessed 1 Feb 2015

  2. Anderson, P., Bijani, S., Vichos, A.: Multi-agent negotiation of virtual machine migration using the lightweight coordination calculus. In: Jezic, G., Kusek, M., Nguyen, N.-T., Howlett, R.J., Jain, L.C. (eds.) 6th KES International Conference on Agent and Multi-agent Systems: Technologies and Applications. Lecture Notes in Computer Science, vol. 7327, pp. 124–133. Springer, Berlin (2012)

    Chapter  Google Scholar 

  3. Andreolini, M., Casolari, S., Colajanni, M., Messori, M.: Dynamic load management of virtual machines in cloud architectures. In: Avresky, D.R., Diaz, M., Bode, A., Ciciani, B., Dekel, E. (eds.) Cloud Computing. Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, vol. 34, pp. 201–214. Springer, Berlin (2010)

    Chapter  Google Scholar 

  4. Bansal, N., Lodi, A., Sviridenko, M.: A tale of two dimensional bin packing. In: Proceedings of the 46th Annual IEEE Symposium on Foundations of Computer Science, pp. 657–666 (2005)

  5. Bellifemine, F., Caire, G., Poggi, A., Rimassa, G.: JADE—a white paper. EXP Search Innov. 3(3), 6–19 (2003)

    Google Scholar 

  6. Bobroff, N., Kochut, A., Beaty K.: Dynamic placement of virtual machines for managing sla violations. In: Proceedings of the 10th IFIP/IEEE International Symposium on Integrated Network Management, pp. 119–128 (2007)

  7. Breitgand, D., Marashini, A., Tordsson, J.: Policy-driven service placement optimization in federated clouds. Technical Report, IBM Research Division (2011)

  8. Buyya, R., Beloglazov, A., Abawajy, J.: Energy-efficient management of data center resources for cloud computing: a vision, architectural elements, and open challenges. In: Proceedings of the 2010 International Conference on Parallel and Distributed Processing Techniques and Application, pp. 12–15 (2010)

  9. Canturk, I., Chengwei, W., Bhatt, C., Shanmuganathan, G., Holler, A.M.: Process demand prediction for distributed power and resource management. U.S. Patent Application 13/281,234, 2011

  10. Chekuri, C., Khanna, S.: On multi-dimensional packing problems. SIAM J. Comput. 33(4), 837–851 (2014)

    Article  MathSciNet  Google Scholar 

  11. Cisco systems: Cisco USB B440 M2 high-performance blade server data sheet. http://www.cisco.com/en/US/prod/collateral/ps10265/ps10280/ps11584/data_sheet_c78-646960.html (2015). Accessed 1 Feb 2015

  12. Clark, C., Fraser, K., Hand, S., Hansen, J.G., Jul, E., Limpach, C., Pratt, I., Warfield, A.: Live migration of virtual machines. In: Proceedings of the 2nd Conference on Symposium on Networked Systems Design & Implementation, USENIX Association, vol. 2, pp. 273–286 (2005)

  13. Coffman, E.G., Garey, M.R., Johnson, D.S., Tarjan, R.E.: Performance bounds for level-oriented two-dimensional packing algorithms. SIAM J. Comput. 9(4), 808–826 (1980)

    Article  MathSciNet  MATH  Google Scholar 

  14. Daniels, J.: Server virtualization architecture and implementation. Crossroads 16(1), 8–12 (2009)

    Article  MathSciNet  Google Scholar 

  15. Das, B., Bharghavan, V.: Routing in ad-hoc networks using minimum connected dominating sets. In: Proceedings of the IEEE International Conference on Communications, pp. 376–380 (1997)

  16. Gutierrez-Garcia, J.O., Ramirez-Nafarrate, A.: Policy-based agents for virtual machine migration in cloud data centers. In: Proceedings of the IEEE International Conference on Services Computing, pp. 603–610 (2013)

  17. Gutierrez-Garcia, J.O., Sim, K.M.: A family of heuristics for agent-based elastic cloud bag-of-tasks concurrent scheduling. Future Gener. Comput. Sys. 29(7), 1682–1699 (2013)

    Article  Google Scholar 

  18. Gutierrez-Garcia, J.O., Sim, K.M.: Agent-based Cloud Service Composition. Appl. Intell. 38(3), 436–464 (2013)

    Article  Google Scholar 

  19. Hyser, C., Mckee, B., Gardner, R., Watson, B.J.: Autonomic virtual machine placement in the data center. Hewlett Packard Laboratories, Technical Report HPL-2007-189, 2007–189 (2007)

  20. Ji, M., Waldspurger, C.A., Zedlewski, J.: Method and system for determining a cost-benefit metric for potential virtual machine migrations. U.S. Patent No. 8,095,929, 2012. Washington, DC: U.S. Patent and Trademark Office

  21. Kerr, A., Diamos, G., Yalamanchili, S.: A characterization and analysis of GPGPU kernels. Technical Report GIT-CERCS-09-06, Georgia Institute of Technology (2009)

  22. Kleiminger, W., Kalyvianaki, E., Pietzuch, P.: Balancing load in stream processing with the cloud. In: Proceedings of the 2011 IEEE 27th International Conference on Data Engineering Workshops, pp. 16–21 (2011)

  23. Lang, W., Patel, J.M., Naughton, J.F.: On energy management, load balancing and replication. ACM SIGMOD Rec. 38(4), 35–42 (2010)

    Article  Google Scholar 

  24. Lin, C.C., Liu, P., Wu, J.J.: Energy-efficient virtual machine provision algorithms for cloud systems. In: Proceedings of the 2011 Fourth IEEE International Conference on Utility and Cloud Computing, pp. 81–88 (2011)

  25. Liu, H., Xu, C.Z., Jin, H., Gong, J., Liao, X.: Performance and energy modeling for live migration of virtual machines. In: Proceedings of the 20th International Symposium on High Performance Distributed Computing, pp. 171–182 (2011)

  26. Metri, G., Srinivasaraghavan, S., Shi, W., Brockmeyer, M.: Experimental analysis of application specific energy efficiency of data centers with heterogeneous servers. In: Proceedings of IEEE 5th International Conference on Cloud Computing, pp. 786–793 (2012)

  27. Montgomery, D.C., Jennings, C.L., Kulahci, M.: Introduction to Time Series Analysis and Forecasting. Wiley, Hoboken (2011)

    Google Scholar 

  28. Patel, P., Bansal, D., Yuan, L., Murthy, A., Greenberg, A., Maltz, D. A., Kern, R., Kumar, H., Zikos, M., Wu, H., Kim, C., Karri, N.: Ananta: Cloud scale load balancing. In: Proceedings of the ACM Conference on SIGCOMM, pp. 207–218 (2013)

  29. Red Hat: Cluster general properties—administration guide. https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Virtualization/3.1/html/Administration_Guide/Cluster_general_entities.html (2015). Accessed 1 Feb 2015

  30. Red Hat: Monitoring—administration guide. https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Virtualization_for_Servers/2.2/html/Administration_Guide/Administration_Guide-Monitoring.html (2015). Accessed 1 Feb 2015

  31. Red Hat: Red Hat Enterprise Virtualization 3.2—explanation of settings and controls in the Edit cluster policy window. https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Virtualization/3.1/html/Administration_Guide/Cluster_general_entities.html (2015). Accessed 1 Feb 2015

  32. Red Hat: Red Hat Enterprise Virtualization 3.2—technical reference guide. https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Virtualization/3.2/html/Technical_Reference_Guide/index.html (2015). Accessed 1 Feb 2015

  33. Ren, G., Tune, E., Moseley, T., Shi, Y., Rus, S., Hundt, R.: Google-wide profiling: a continuous profiling infrastructure for data centers. IEEE Micro 30(4), 65–79 (2010)

    Article  Google Scholar 

  34. Ren, X., Lin, R., Zou, H.: A dynamic load balancing strategy for cloud computing platform based on exponential smoothing forecast. In: Proceedings of the 2011 IEEE International Conference on Cloud Computing and Intelligence Systems, pp. 220–224 (2011)

  35. Siebert, E.: Understanding VMware ESX virtual machine physical and virtual hardware. http://searchvmware.techtarget.com/tip/Understanding-VMware-ESX-virtual-machine-physical-and-virtual-hardware (2015). Accessed 1 Feb 2015

  36. Sim, K.M.: Agent-based cloud computing. IEEE T. Serv. Comput. 5(4), 564–577 (2012)

    Article  Google Scholar 

  37. Skiena, S.: The Algorithm Design Manual: Text, vol. 1. Springer, Berlin (1998)

    Google Scholar 

  38. SPEC: All published SPECvirt_sc2010 performance results SPEC. http://www.spec.org/virt_sc2010/results/specvirt_sc2010_perf.html (2015). Accessed 1 Feb 2015

  39. Speitkamp, B., Bichler, M.: A mathematical programming approach for server consolidation problems in virtualized data centers. IEEE T. Serv. Comput. 3(4), 266–278 (2010)

    Article  Google Scholar 

  40. Tsai, Y.L., Huang, K.C., Chang, H.Y., Ko, J., Wang, E.T., Hsu, C.H.: Scheduling multiple scientific and engineering workflows through task clustering and best-fit allocation. In: Proceedings of the 2012 IEEE Eighth World Congress on Services, pp. 1–8 (2012)

  41. Vaquero, L.M., Rodero-Merino, L., Caceres, J., Lindner, M.: A break in the clouds: towards a cloud definition. ACM SIGCOMM Computer Communication Review 39(1), 50–55 (2008)

    Article  Google Scholar 

  42. VMware: Distributed resource scheduler—recommendation page. http://pubs.vmware.com/vsphere-4-esx-vcenter/index.jsp#viewing_drs_cluster_information/c_the_drs_recommendations_page.html (2015). Accessed 1 Feb 2015

  43. VMware: VMware Infrastructure—Resource management with VMware distributed resource scheduler. http://www.vmware.com/pdf/vmware_drs_wp.pdf (2015). Accessed 1 Feb 2015

  44. Von Laszewski, G., Diaz, J., Wang, F., Fox, G.C.: Comparison of multiple cloud frameworks. In: Proceedings of the 2012 IEEE 5th International Conference on Cloud Computing, pp. 734–741 (2012)

  45. Voorsluys, W., Broberg, J., Venugopal, S., Buyya, R.: Cost of virtual machine live migration in clouds: a performance evaluation. Cloud Computing, pp. 254–265. Springer, Berlin (2009)

    Chapter  Google Scholar 

  46. Wood, T., Shenoy, P., Venkataramani, A., Yousif, M.: Black-box and gray-box strategies for virtual machine migration. In: Proceedings of the Fourth USENIX Symposium on Networked Systems Design & Implementation, pp. 701–708 (2007)

  47. Wu, Y., Yuan, Y., Yang, G., Zheng, W.: Load prediction using hybrid model for computational grid. In: Proceedings of the 8th IEEE/ACM International Conference on Grid Computing, pp. 235–242 (2007)

  48. Xing, L.: A self-management approach to service optimization and system integrity through multi-agent systems. Master’s Thesis, University of Oslo (2008)

  49. Zhang, H., Jiang, F., Yoshihira, K., Chen, H., Saxena, A.: Intelligent workload factoring for a hybrid cloud computing model. In: Proceedings of the 2009 World Conference on Services-I, pp. 701–708 (2009)

Download references

Acknowledgments

The authors would like to thank the Editor-in-Chief and the anonymous referees for their comments and suggestions. This work has been supported by Asociación Mexicana de Cultura A.C.

Author information

Authors and Affiliations

  1. Department of Computer Science, Instituto Tecnológico Autónomo de México, 1 Rio Hondo St., Progreso Tizapan, 01080, Mexico City, DF, Mexico

    J. Octavio Gutierrez-Garcia

  2. Department of Industrial & Operations Engineering, Instituto Tecnológico Autónomo de México, 1 Rio Hondo St., Progreso Tizapan, 01080, Mexico City, DF, Mexico

    Adrian Ramirez-Nafarrate

Authors
  1. J. Octavio Gutierrez-Garcia
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Adrian Ramirez-Nafarrate
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to J. Octavio Gutierrez-Garcia.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gutierrez-Garcia, J.O., Ramirez-Nafarrate, A. Agent-based load balancing in Cloud data centers. Cluster Comput 18, 1041–1062 (2015). https://doi.org/10.1007/s10586-015-0460-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10586-015-0460-x

Keywords

Search

Navigation