Skip to main content
SpringerLink
Log in

A reinforcement learning-based load balancing algorithm for fog computing

  • Published:
Telecommunication Systems Aims and scope Submit manuscript

Abstract

Fog computing is a developing paradigm for bringing cloud computing capabilities closer to end-users. Fog computing plays an important role in improving resource utilization and decreasing delay for internet of things (IoT) applications. At the same time, it faces many challenges, including challenges related to energy consumption, scheduling and resource overload. Load balancing helps to reduce delay, increase user satisfaction, and also increase system efficiency by efficiently and fairly allocation of tasks among computing resources. Fair load distribution among fog nodes is a difficult challenge due to the increasing number of IoT devices. In this research, we suggested a new approach for fair load distribution in fog environment. The Q-learning algorithm-based load balancing method is executed as the proposed approach in the fog layer. The objective of this method is to simultaneously improve the load balancing and delay. In this technique, the fog node uses reinforcement learning to choose whether to handle a task it receives via IoT devices directly, or whether to send it to a nearby fog node or the cloud. The simulation findings demonstrate that our approach results a suitable technique for fair load distribution among fog nodes, which improves the delay, run time, network utilization, and standard deviation of load on nodes than other compared techniques. In this way, in the case where the number of fog nodes is considered to be 4, the delay in the proposed method is reduced by around 8.44% in comparison to the load balancing and optimization strategy (LBOS) method, 26.65% in comparison to the secure authentication and load balancing (SALB) method, 29.15% in comparison to the proportional method, 7.75% in comparison to the fog cluster-based load-balancing (FCBLB) method, and 36.22% in comparison to the random method. In the case where the number of fog nodes is considered to be 10, the delay in the proposed method is reduced by around 13.80% in comparison to the LBOS method, 29.84% in comparison to the SALB method, 32.23% in comparison to the proportional method, 13.34% in comparison to the FCBLB method, and 39.1% in comparison to the Random method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Data availability

The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.

Notes

  1. Quality of Experience.

  2. Quality of Service.

  3. Electro Encephalo Gram.

  4. Load Balancing and Optimization Strategy.

  5. Secure Authentication and Load Balancing.

  6. Fog Cluster-Based Load-Balancing.

References

  1. Martinez, I., Hafid, A. S., & Jarray, A. (2021). Design, resource management, and evaluation of fog computing systems: a survey. IEEE Internet of Things Journal, 8(4), 2494–2516.

    Article  Google Scholar 

  2. Yousefpour, A., Fung, C., Nguyen, T., Kadiyala, K., Jalali, F., Niakanlahiji, A., Kong, J., & Jue, J. P. (2019). All one needs to know about fog computing and related edge computing paradigms: A complete survey. Journal of Systems Architecture, 98, 289–330.

    Article  Google Scholar 

  3. Kashani, M. H., & Mahdipour, E. (2023). Load balancing algorithms in fog computing. IEEE Transactions on Services Computing, 16, 1505–1521.

    Article  Google Scholar 

  4. Kashyap, V., & Kumar, A. (2022). Load balancing techniques for fog computing environment: Comparison, taxonomy, open issues, and challenges. Concurrency and Computation: Practice and Experience, 34(23), e7183.

    Article  Google Scholar 

  5. Jamil, B., Ijaz, H., Shojafar, M., Munir, K., & Buyya, R. (2022). Resource allocation and task scheduling in fog computing and internet of everything environments: a taxonomy, review, and future directions. ACM Computing Surveys, 54, 1–38. https://doi.org/10.1145/3513002

    Article  Google Scholar 

  6. Batra, S., Anand, D., & Singh, A. (2022). A brief overview of load balancing techniques in fog computing environment. In 2022 6th international conference on trends in electronics and informatics (ICOEI). IEEE (pp. 886–891).

  7. Hamid, L., Jadoon, A., & Asghar, H. (2022). Comparative analysis of task level heuristic scheduling algorithms in cloud computing. The Journal of Supercomputing, 78, 12931–12949.

    Article  Google Scholar 

  8. Chowdhury, S., & Katangur, A. (2022). Threshold based load balancing algorithm in cloud computing. In 2022 IEEE international conference on joint cloud computing (JCC). IEEE (pp. 23–28).

  9. Katangur, A., Akkaladevi, S., & Vivekanandhan, S. (2022). Priority weighted round robin algorithm for load balancing in the cloud. In 2022 IEEE 7th international conference on smart cloud (SmartCloud). IEEE (pp. 230–235).

  10. Al-Amodi, S., Patra, S. S., Bhattacharya, S., Mohanty, J. R., Kumar, V., & Barik, R. K. (2022). Meta-heuristic algorithm for energy-efficient task scheduling in fog computing. Recent trends in electronics and communication (pp. 915–925). Springer.

    Chapter  Google Scholar 

  11. AlShathri, S. I., Chelloug, S. A., & Hassan, D. S. (2022). Parallel meta-heuristics for solving dynamic offloading in fog computing. Mathematics, 10(8), 1258.

    Article  Google Scholar 

  12. He, J. (2022). Cloud computing load balancing mechanism taking into account load balancing ant colony optimization algorithm. Computational Intelligence and Neuroscience. https://doi.org/10.1155/2022/3120883

    Article  Google Scholar 

  13. Ijeoma, C. C., Inyiama, P., Samuel, A., Okechukwu, O. M., & Chinedu, A. D. (2022). Review of hybrid load balancing algorithms in cloud computing environment. arXiv:2202.13181.

  14. Khan, M. S. A., & Santhosh, R. (2022). Task scheduling in cloud computing using hybrid optimization algorithm. Soft Computing, 26(23), 13069–13079.

    Article  Google Scholar 

  15. Talaat, F. M., Ali, H. A., Saraya, M. S., & Saleh, A. I. (2022). Effective scheduling algorithm for load balancing in fog environment using CNN and MPSO. Knowledge and Information Systems, 64(3), 773–797.

    Article  Google Scholar 

  16. Gures, E., Shayea, I., Ergen, M., Azmi, M. H., & El-Saleh, A. A. (2022). Machine learning-based load balancing algorithms in future heterogeneous networks: a survey. IEEE Access, 10, 37689–37717.

    Article  Google Scholar 

  17. Sheikhpour, R., Sarram, M. A., Gharaghani, S., & Chahooki, M. A. Z. (2017). A survey on semi-supervised feature selection methods. Pattern Recognition, 64, 141–158.

    Article  Google Scholar 

  18. Tran-Dang, H., Bhardwaj, S., Rahim, T., Musaddiq, A., & Kim, D.-S. (2022). Reinforcement learning based resource management for fog computing environment: Literature review, challenges, and open issues. Journal of Communications and Networks, 24(1), 83–98.

    Article  Google Scholar 

  19. Prudencio, R. F., Maximo, M. R., & Colombini, E. L. (2022). A survey on offline reinforcement learning: taxonomy, review, and open problems. arXiv:2203.01387.

  20. Xu, Y., Xu, W., Wang, Z., Lin, J., & Cui, S. (2019). Load balancing for ultradense networks: A deep reinforcement learning-based approach. IEEE Internet of Things Journal, 6(6), 9399–9412.

    Article  Google Scholar 

  21. Li, S. E. (2023). Deep reinforcement learning. Reinforcement learning for sequential decision and optimal control (pp. 365–402). Springer.

    Chapter  Google Scholar 

  22. Wang, X., Wang, S., Liang, X., Zhao, D., Huang, J., Xin, X., Dai, B., & Miao, Q. (2022). Deep reinforcement learning: a survey. IEEE Transactions on Neural Networks and Learning Systems. https://doi.org/10.1109/TNNLS.2022.3207346

    Article  Google Scholar 

  23. Singh, J., Singh, P., Amhoud, E. M., & Hedabou, M. (2022). Energy-efficient and secure load balancing technique for SDN-enabled fog computing. Sustainability, 14(19), 12951.

    Article  Google Scholar 

  24. Phan, L. A., Nguyen, D. T., Lee, M., Park, D. H., & Kim, T. (2021). Dynamic fog-to-fog offloading in SDN-based fog computing systems. Future Generation Computer Systems, 117, 486–497.

    Article  Google Scholar 

  25. Belkout, N. E., Zeraoulia, K., Shahzad, M. N., Liu, L., & Yuan, B. (2022). A load balancing and routing strategy in fog computing using deep reinforcement learning. In 2022 international conference on electrical, computer and energy technologies (ICECET). IEEE (pp. 1–8).

  26. Singh, P., Kaur, R., Rashid, J., Juneja, S., Dhiman, G., Kim, J., & Ouaissa, M. (2022). A fog-cluster based load-balancing technique. Sustainability, 14(13), 7961.

    Article  Google Scholar 

  27. Nair, B., & Bhanu, S. (2022). A reinforcement learning algorithm for rescheduling preempted tasks in fog nodes. Journal of Scheduling, 25, 547–565.

    Article  Google Scholar 

  28. Baek, J., & Kaddoum, G. (2021). Heterogeneous task offloading and resource allocations via deep recurrent reinforcement learning in partial observable multifog networks. IEEE Internet of Things Journal, 8(2), 1041–1056.

    Article  Google Scholar 

  29. Divya, V., & Sri, R. L. (2019). ReTra: reinforcement based traffic load balancer in fog based network. In 2019 10th international conference on computing, communication and networking technologies (ICCCNT), 2019.

  30. Talaat, F. M., Saraya, M. S., Saleh, A. I., Ali, H. A., & Ali, S. H. (2020). A load balancing and optimization strategy (LBOS) using reinforcement learning in fog computing environment. Journal of Ambient Intelligence and Humanized Computing, 11(11), 4951–4966.

    Article  Google Scholar 

  31. Kadhim, A. J., & Naser, J. I. (2021). Proactive load balancing mechanism for fog computing supported by parked vehicles in IoV-SDN. China Communications, 18(2), 271–289.

    Article  Google Scholar 

  32. Sharma, S., & Saini, H. (2019). A novel four-tier architecture for delay aware scheduling and load balancing in fog environment. Sustainable Computing: Informatics and Systems, 24, 100355.

    Google Scholar 

  33. Xu, X., Fu, S., Cai, Q., Tian, W., Liu, W., Dou, W., Sun, X., & Liu, A. X. (2018). Dynamic resource allocation for load balancing in fog environment. Wireless Communications and Mobile Computing, 2018, 6421607.

    Article  Google Scholar 

  34. Beraldi, R., Canali, C., Lancellotti, R., & Mattia, G. P. (2020). A random walk based load balancing algorithm for fog computing. In 2020 Fifth international conference on fog and mobile edge computing (FMEC), 2020.

  35. Manju, A. B., & Sumathy, S. (2019). Efficient load balancing algorithm for task preprocessing in fog computing environment. Smart intelligent computing and applications (pp. 291–298). Springer.

    Chapter  Google Scholar 

  36. Talaat, F. M., Ali, S. H., Saleh, A. I., & Ali, H. A. (2019). Effective load balancing strategy (ELBS) for real-time fog computing environment using fuzzy and probabilistic neural networks. Journal of Network and Systems Management, 27(4), 883–929.

    Article  Google Scholar 

  37. Pradhan, A., Bisoy, S. K., Kautish, S., Jasser, M. B., & Mohamed, A. W. (2022). Intelligent decision-making of load balancing using deep reinforcement learning and parallel PSO in cloud environment. IEEE Access, 10, 76939–76952.

    Article  Google Scholar 

  38. Puthal, D., Ranjan, R., Nanda, A., Nanda, P., Jayaraman, P. P., & Zomaya, A. Y. (2019). Secure authentication and load balancing of distributed edge data centers. Journal of Parallel and Distributed Computing, 124, 60–69.

    Article  Google Scholar 

  39. Tellioglu, I., & Mantar, H. A. (2009). A proportional load balancing for wireless sensor networks. In 2009 third international conference on sensor technologies and applications (pp. 514–519).

  40. Houidi, O., Zeghlache, D., Perrier, V., Quang, P. T. A., Huin, N., Leguay, J., & Medagliani, P. (2022). Constrained deep reinforcement learning for smart load balancing. In 2022 IEEE 19th annual consumer communications & networking conference (CCNC). IEEE (pp. 207–215).

  41. Lu, H., Gu, C., Luo, F., Ding, W., & Liu, X. (2020). Optimization of lightweight task offloading strategy for mobile edge computing based on deep reinforcement learning. Future Generation Computer Systems, 102, 847–861.

    Article  Google Scholar 

  42. Zhu, X. k., Zhang, Q., Liu, L., Cheng, T., Yao, S., Zhou, W., & He, J. (2019). DLB: deep learning based load balancing. arXiv:1910.08494.

  43. Alvi, A. N., Javed, M. A., Hasanat, M. H. A., Khan, M. B., Saudagar, A. K. J., Alkhathami, M., & Farooq, U. (2022). Intelligent task offloading in fog computing based vehicular networks. Applied Sciences, 12(9), 4521.

    Article  Google Scholar 

  44. Tu, Y., Chen, H., Yan, L., & Zhou, X. (2022). Task offloading based on LSTM prediction and deep reinforcement learning for efficient edge computing in IoT. Future Internet, 14(2), 30.

    Article  Google Scholar 

  45. Gupta, H., Dastjerdi, A. V., Ghosh, S. K., & Buyya, R. (2017). iFogSim: A toolkit for modeling and simulation of resource management techniques in the internet of things, edge and fog computing environments. Software-Practice and Experience, 47(9), 1275–1296.

    Article  Google Scholar 

  46. Mahmud, R., & Buyya, R. (2019). Modeling and simulation of fog and edge computing environments using iFogSim toolkit. Fog and edge computing (pp. 433–465). Wiley.

    Chapter  Google Scholar 

  47. Tahmasebi-Pouya, N., Sarram, M. A., & Mostafavi, S. (2022). A blind load-balancing algorithm (BLBA) for distributing tasks in fog nodes. Wireless Communications and Mobile Computing. https://doi.org/10.1155/2022/1533949

    Article  Google Scholar 

Download references

Funding

The authors declare that no funds, grants, or other support were received during the preparation of this manuscript.

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Yazd University, Yazd, Iran

    Niloofar Tahmasebi-Pouya, Mehdi Agha Sarram & Seyedakbar Mostafavi

Authors
  1. Niloofar Tahmasebi-Pouya
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mehdi Agha Sarram
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Seyedakbar Mostafavi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Seyedakbar Mostafavi.

Ethics declarations

Conflict of interest

The authors have no relevant financial or non-financial interests to disclose.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tahmasebi-Pouya, N., Sarram, M.A. & Mostafavi, S. A reinforcement learning-based load balancing algorithm for fog computing. Telecommun Syst 84, 321–339 (2023). https://doi.org/10.1007/s11235-023-01049-7

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11235-023-01049-7

Keywords

Search

Navigation