Skip to main content

Advertisement

SpringerLink
Log in

Novel dynamic load balancing algorithm for cloud-based big data analytics

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

A Correction to this article was published on 10 September 2021

This article has been updated

Abstract

Big data analytics in cloud environments introduces challenges such as real-time load balancing besides security, privacy, and energy efficiency. This paper proposes a novel load balancing algorithm in cloud environments that performs resource allocation and task scheduling efficiently. The proposed load balancer reduces the execution response time in big data applications performed on clouds. Scheduling, in general, is an NP-hard problem. Our proposed algorithm provides solutions to reduce the search area that leads to reduced complexity of the load balancing. We recommend two mathematical optimization models to perform dynamic resource allocation to virtual machines and task scheduling. The provided solution is based on the hill-climbing algorithm to minimize response time. We evaluate the performance of proposed algorithms in terms of response time, turnaround time, throughput metrics, and request distribution with some of the existing algorithms that show significant improvements.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Change history

References

  1. Klous S, Wielaard N (2016) We are big data: the future of the information society

  2. Skourletopoulos G, Mavromoustakis C, Mastorakis G, Batalla JM, Dobre C, Panagiotakis S, Pallis E (2017) Big data and cloud computing: a survey of the state-of-the-art and research challenges

  3. Zhang R (2020) The impacts of cloud computing architecture on cloud service performance. J Comput Inf Syst 60:166–174

    Google Scholar 

  4. Chang WL, Laszewski G (2019) NIST big data interoperability framework: volume 8, reference architecture interfaces

  5. Khan S, Shakil KA, Alam M (2017) Big data computing using cloud-based technologies, challenges and future perspectives. ArXiv, abs/1712.05233

  6. Wang L, Jones R (2020) Big data analytics in cyber security: network traffic and attacks. J Comput Inf Syst, pp 1–8

  7. Khatibi E, Mirtaheri SL (2019) A dynamic data dissemination mechanism for Cassandra NoSQL data store. J Supercomput 75(11):7479–7496. https://doi.org/10.1007/s11227-019-02959-7

    Article  Google Scholar 

  8. Neves PC, Schmerl B, Cámara J, Bernardino J (2016) Big data in cloud computing. Features Issues, IoTBD

  9. Xiong H, Wang Y, Li W, Chen C (2019) Flexible, efficient, and secure access delegation in cloud computing. ACM Trans Manag. Inf Syst 10(2):1–2

    Google Scholar 

  10. Yadav V, Yadav MP, Yadav DK (2012) Reliable task allocation in heterogeneous distributed system with random node failure: load sharing approach. Int Conf Comput Sci 2012:187–192

    Google Scholar 

  11. Patel N, Chauhan S (2014) A survey on load balancing and scheduling in cloud computing

  12. Singh A, Juneja D, Malhotra M (2015) Autonomous agent based load balancing algorithm in cloud computing. Procedia Comput Sci 45:832–841

    Article  Google Scholar 

  13. Mata-Toledo RA, Madison J, Gupta P (2010) Green data center: How green can we perform?. J Technol Res

  14. Chen Y, Argentinis JD, Weber G (2016) IBM Watson: how cognitive computing can be applied to big data challenges in life sciences research. Clin Ther 38(4):688–701

    Article  Google Scholar 

  15. Shah N, Farik M (2015) Static load balancing algorithms in cloud computing: challenges and solutions. Int J Sci Technol Res 4:365–367

    Google Scholar 

  16. Fox G, Qiu J, Jha S, Ekanayake S, Kamburugamuve S (2015) Big data. Simulations and HPC convergence, WBDB

  17. Lohr Steve (2012) The age of big data. New York Times

  18. Kansal N, Chana I (2012) Cloud load balancing techniques a step towards green computing. Int J Comput Sci Issues

  19. Baliga J, Ayre R, Hinton K, Tucker R (2011) Green cloud computing: balancing energy in processing, storage, and transport. Proc IEEE 99:149–167

    Article  Google Scholar 

  20. Hwang K, Dongarra J, Fox G (2011) Distributed and cloud computing: from parallel processing to the internet of things. Morgan Kaufmann

  21. Ghomi EJ, Rahmani A, Qader N (2017) Load-balancing algorithms in cloud computing: a survey. J Netw Comput Appl 88:50–71

    Article  Google Scholar 

  22. Rastogi G, Sushil R (2015) Analytical literature survey on existing load balancing schemes in cloud computing. Int Conf Green Comput Internet Things ICGCIoT 2015:1506–1510

    Google Scholar 

  23. Wang S, Yan K, Liao W, Wang S (2010) Towards a load balancing in a three-level cloud computing network. In: 2010 3rd International Conference on Computer Science and Information Technology, 1:108–113

  24. Mirtaheri SL, Grandinetti L (2017) Dynamic load balancing in distributed exascale computing systems. Cluster Comput 20:3677–3689

    Article  Google Scholar 

  25. Kumar M, Sharma S (2017) Dynamic load balancing algorithm for balancing the workload among virtual machine in cloud computing. Procedia Comput Sci 115:322–329

    Article  Google Scholar 

  26. Karthick A, Ramaraj E, Subramanian R (2014) An efficient multi queue job scheduling for cloud computing. World Cong Comput Commun Technol 2014:164–166

    Google Scholar 

  27. Kaur S, Kaur G (2015) A review of load balancing strategies for distributed systems. Int J Comput Appl 121:45–47

    MATH  Google Scholar 

  28. Mell P, Grance T (2011) The NIST definition of cloud computing

  29. Sharma M, Bhatia J (2013) A review on different approaches for load balancing in computational grid. J Global Res Comput Sci 4:82–85

    Google Scholar 

  30. Liu G, Li J, Xu J (2013) Liu G, Li J, Xu J (2013) An improved min–min algorithm in cloud computing. In: Proceedings of the 2012 International Conference of Modern Computer Science and Applications, pp 47–52

  31. El-Zoghdy SF, Ghoniemy S (2014) A survey of load balancing in high-performance distributed computing systems. Int J Adv Comput Res

  32. Elzeki OM, Reshad M, Elsoud MA (2012) Improved max–min algorithm in cloud computing. Int J Comput Appl 50:22–27

    Google Scholar 

  33. Sharma N, Tyagi S, Atri S (2017) A comparative analysis of min–min and max–min algorithms based on the makespan parameter. Int J Adv Res Comput Sci 8:1038–1041

    Google Scholar 

  34. Kumar R, Prashar T (2015) Performance analysis of load balancing algorithms in cloud computing. Int J Comput Appl 120(7):19–27

    Google Scholar 

  35. Domanal SG, Reddy GR (2014) Optimal load balancing in cloud computing by efficient utilization of virtual machines. Sixth Int Conf Commun Syst Netw COMSNETS 2014:1–4

    Google Scholar 

  36. Li J, Ma T, Tang M, Shen W, Jin Y (2017) Improved FIFO scheduling algorithm based on fuzzy clustering in cloud computing. Information 8:25

    Article  Google Scholar 

  37. Hamdani M, Aklouf Y, Bouarara HA (2019) Improved fuzzy load-balancing algorithm for cloud computing system. In: Proceedings of the 9th International Conference on Information Systems and Technologies

  38. Daraghmi E, Yuan S (2015) A small world based overlay network for improving dynamic load-balancing. J Syst Softw 107:187–203

    Article  Google Scholar 

  39. Saleh H, Nashaat H, Saber W, Harb H (2019) IPSO task scheduling algorithm for large scale data in cloud computing environment. IEEE Access 7:5412–5420

    Article  Google Scholar 

  40. Sanaj MS, Prathap P (2020) Nature inspired chaotic squirrel search algorithm (CSSA) for multi objective task scheduling in an IAAS cloud computing atmosphere. Eng Sci Technol Int J 23:891–902

    Google Scholar 

  41. Samadi Y, Zbakh M (2017) Threshold-based load balancing algorithm for big data on a cloud environment. BDCA’17

  42. Manikandan N, Pravin A (2019) LGSA: hybrid task scheduling in multi objective functionality in cloud computing environment 3D. Research 10:1–16

    Google Scholar 

  43. Mousavi, S., Mosavi, A., Várkonyi-Káczy, A. (2017). A load balancing algorithm for resource allocation in cloud computing

  44. Vashishth V, Chhabra A, Sood A (2017) A predictive approach to task scheduling for big data in cloud environments using classification algorithms. In: 2017 7th International Conference on Cloud Computing, Data Science and Engineering—Confluence, pp 188–192

  45. Lagwal M, Bhardwaj N (2017) Load balancing in cloud computing using genetic algorithm. Int Conf Intell Comput Control Syst ICICCS 2017:560–565

    Google Scholar 

  46. Ebadifard F, Babamir SM, Barani S (2020) A dynamic task scheduling algorithm improved by load balancing in cloud computing. In: 2020 6th International Conference on Web Research (ICWR)

  47. Tadi AA, Khayyambashi M, Farsani HK (2020) OASM: An overload-aware workload scheduling method for cloud computing based on biogeographical optimization. Int J Netw Manag, p 30

  48. Kaur G, Bala A (2021) OPSA: an optimized prediction based scheduling approach for scientific applications in cloud environment. Clust Comput, pp 1–20

  49. Russell S, Norvig P (2011) Artificial intelligence: a modern approach, 3rd edn. Artif Intell 175:122–125

  50. Wolpert D, Macready W (1995) No free lunch theorems for search

  51. Wolpert D, Macready W (1997) No free lunch theorems for optimization. IEEE Trans Evol Comput 1:67–82

    Article  Google Scholar 

  52. Lattimore T, Hutter M (2011) No Free lunch versus Occam’s razor in supervised learning. Algorithmic probability and friends. Bayes Predict Artif Intell, pp. 223–235

  53. Yang X (2012) Swarm-based metaheuristic algorithms and no-free-lunch theorems

  54. Calheiros R, Ranjan R, Beloglazov A, Rose C, Buyya R (2011) CloudSim: a toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms. Pract Exp Softw 41

Download references

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, Faculty of Engineering, Kharazmi University, Tehran, Iran

    Arman Aghdashi & Seyedeh Leili Mirtaheri

Authors
  1. Arman Aghdashi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Seyedeh Leili Mirtaheri
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Seyedeh Leili Mirtaheri.

Additional information

The original online version of this article was revised: In this article ref. 7 was incorrect.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Aghdashi, A., Mirtaheri, S.L. Novel dynamic load balancing algorithm for cloud-based big data analytics. J Supercomput 78, 4131–4156 (2022). https://doi.org/10.1007/s11227-021-04024-8

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-021-04024-8

Keywords

Search

Navigation