Skip to main content
SpringerLink
Log in

An autonomous architecture based on reinforcement deep neural network for resource allocation in cloud computing

  • Regular Paper
  • Published:
Computing Aims and scope Submit manuscript

Abstract

Today, cloud computing technology has attracted the attention of many researchers. According to the needs of users to quickly execute requests and provide quality services, optimal allocation of resources and timing of task execution between virtual machines in cloud computing are of great importance. One of the important challenges that cloud service providers face is the effective management of resources by physical infrastructure. Therefore, in this paper, an autonomous system based on the Clipped Double Deep Q-Learning (CDDQL) Algorithm and the meta-heuristic Particle Swarm Optimization (PSO) for resource allocation is proposed in the Fog-cloud computing infrastructure. The PSO algorithm is used to prioritize the tasks and CDDQL is used as the core of the autonomous system (Auto-CDDQL) to allocate the desired VM resources to the tasks. The proposed Auto-CDDQL is implemented in the Fog and performs this process autonomously. By evaluating the results, it was observed that the amount of Make Span, response time, task completion, resource utilization, and energy consumption rate in the proposed AutoCDDQL on the c-hilo dataset, compared to the FCFS, RR, and PBTS methods, are significantly improved.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Lu G, Zeng WH (2014) Cloud computing survey. Appl Mech Mater 530:650–661

    Article  Google Scholar 

  2. Ghomi EJ, Rahmani AM, Qader NN (2017) Load-balancing algorithms in cloud computing: a survey. J Netw Comput Appl 88:50–71

    Article  Google Scholar 

  3. Aslam S, Shah MA (2015) Load balancing algorithms in cloud computing: a survey of modern techniques. In: 2015 National software engineering conference (NSEC), pp 30–35. IEEE

  4. González-Martínez JA, Bote-Lorenzo ML, Gómez-Sánchez E, Cano-Parra R (2015) Cloud computing and education: a state-of-the-art survey. Comput Educ 80:132–151

    Article  Google Scholar 

  5. Mastelic T, Oleksiak A, Claussen H, Brandic I, Pierson JM, Vasilakos AV (2014) Cloud computing: survey on energy efficiency. ACM Comput Surv 47(2):1–36

    Article  Google Scholar 

  6. Mustafa S, Nazir B, Hayat A, Madani SA (2015) Resource management in cloud computing: taxonomy, prospects, and challenges. Comput Electr Eng 47:186–203

    Article  Google Scholar 

  7. https://ieee-dataport.org/documents/dataset-task-scheduling-cloud-using-cloudsim#files

  8. Das R, Inuwa MM (2023) A review on fog computing: Issues, characteristics, challenges, and potential applications. Telematics and Informatics Reports, 100049

  9. Hazra A, Rana P, Adhikari M, Amgoth T (2023) Fog computing for next-generation Internet of Things: fundamental, state-of-the-art and research challenges. Comput Sci Rev 48:100549

    Article  Google Scholar 

  10. Costa B, Bachiega J Jr, de Carvalho LR, Araujo AP (2022) Orchestration in fog computing: a comprehensive survey. ACM Comput Surv 55(2):1–34

    Article  Google Scholar 

  11. Kansal NJ, Chana I (2012) Cloud load balancing techniques: a step towards green computing. IJCSI Int J Comput Sci Issues 9(1):238–246

    Google Scholar 

  12. Kliazovich D, Arzo ST, Granelli F, Bouvry P, Khan SU (2013) e-STAB: energy-efficient scheduling for cloud computing applications with traffic load balancing. In: 2013 IEEE international conference on green computing and communications and IEEE Internet of Things and IEEE cyber, physical and social computing, pp 7–13. IEEE

  13. James J, Verma B (2012) Efficient VM load balancing algorithm for a cloud computing environment. Int J Comput Sci Eng 4(9):1658

    Google Scholar 

  14. Falisha IN, Purboyo TW, Latuconsina R, Robin AR (2018) Experimental model for load balancing in cloud computing using equally spread current execution load algorithm. Int J Appl Eng Res 13(2):1134–1138

    Google Scholar 

  15. Liu G, Li J, Xu J (2013) An improved min-min algorithm in cloud computing. In: Proceedings of the 2012 international conference of modern computer science and applications. Springer, Berlin, pp 47–52

  16. Elzeki OM, Reshad MZ, Elsoud MA (2012) Improved max-min algorithm in cloud computing. Int J Comput Appl 50(12)

  17. Neelima P, Reddy ARM (2020) An efficient load balancing system using adaptive dragonfly algorithm in cloud computing. Clust Comput 23:2891–2899

    Article  Google Scholar 

  18. Ghorashi H, Mirabi M (2020) An effective task scheduling framework for cloud computing using NSGA-II. J Adv Comput Eng Technol 6(3):155–168

    Google Scholar 

  19. Gill SS, Garraghan P, Stankovski V, Casale G, Thulasiram RK, Ghosh SK, Buyya R (2019) Holistic resource management for sustainable and reliable cloud computing: an innovative solution to global challenge. J Syst Softw 155:104–129

    Article  Google Scholar 

  20. Abbasi M, Yaghoobikia M, Rafiee M, Jolfaei A, Khosravi MR (2020) Efficient resource management and workload allocation in fog–cloud computing paradigm in IoT using learning classifier systems. Comput Commun 153:217–228

    Article  Google Scholar 

  21. Buvana M, Loheswaran K, Madhavi K, Ponnusamy S, Behura A, Jayavadivel R (2021) Improved Resource management and utilization based on a fog-cloud computing system with IoT incorporated with classifier systems. Microprocess Microsyst 103815

  22. Praveenchandar J, Tamilarasi A (2021) Dynamic resource allocation with optimized task scheduling and improved power management in cloud computing. J Ambient Intell Humaniz Comput 12:4147–4159

    Article  Google Scholar 

  23. Tuli S, Gill SS, Xu M, Garraghan P, Bahsoon R, Dustdar S, Jennings NR (2022) HUNTER: AI based holistic resource management for sustainable cloud computing. J Syst Softw 84:111124

  24. Jeong B, Baek S, Park S, Jeon J, Jeong YS (2023) Stable and efficient resource management using deep neural network on cloud computing. Neurocomputing 521:99–112

    Article  Google Scholar 

  25. Saad ZM, Mhmood MR (2023) Fog computing system for internet of things: Survey. Texas J Eng Technol 16:1–10

    Google Scholar 

  26. Saif FA, Latip R, Hanapi ZM, Shafinah K (2023) Multi-objective grey wolf optimizer algorithm for task scheduling in cloud-fog computing. IEEE Access 11:20635–20646

    Article  Google Scholar 

  27. Matrouk KM, Matrouk AD (2023) Mobility aware-task scheduling and virtual fog for offloading in IoT-fog-cloud environment. Wireless Personal Communications, 1–36

  28. Hussain T, Yang B, Rahman HU, Iqbal A, Ali F (2022) Improving Source location privacy in social Internet of Things using a hybrid phantom routing technique. Comput Secur 123:102917

    Article  Google Scholar 

  29. Feng Y, Liu F (2022) Resource management in cloud computing using deep reinforcement learning: a survey. In: China aeronautical science and technology youth science forum (pp. 635–643).Springer Nature Singapore: Singapore

  30. Godhrawala H, Sridaran R (2022) Improving architectural reusability for resource allocation framework in futuristic cloud computing using decision tree based multi-objective automated approach. In: International conference on advancements in smart computing and information security, pp 397–415.Springer: Cham

  31. Sabireen H, Neelanarayanan VJIE (2021) A review on fog computing: architecture, fog with IoT, algorithms and research challenges. Ict Express 7(2):162–176

    Article  Google Scholar 

  32. Liu Y, Fieldsend JE, Min G (2017) A framework of fog computing: architecture, challenges, and optimization. IEEE Access 5:25445–25454

    Article  Google Scholar 

  33. Aazam M, Zeadally S, Harras KA (2018) Fog computing architecture, evaluation, and future research directions. IEEE Commun Mag 56(5):46–52

    Article  Google Scholar 

  34. Swarup S, Shakshuki EM, Yasar A (2021) Task scheduling in cloud using deep reinforcement learning. Procedia Comput Sci 184:42–51

    Article  Google Scholar 

  35. Marini F, Walczak B (2015) Particle swarm optimization (PSO): a tutorial. Chemom Intell Lab Syst 149:153–165

    Article  Google Scholar 

  36. Jiang H, Xie J, Yang J (2021) Action candidate based clipped double q-learning for discrete and continuous action tasks. In: Proceedings of the AAAI conference on artificial intelligence, vol 35(9), 7979–7986

  37. Hu P, Dhelim S, Ning H, Qiu T (2017) Survey on fog computing: architecture, key technologies, applications and open issues. J Netw Comput Appl 98:27–42

    Article  Google Scholar 

  38. Aazam M, Huh EN (2015) Fog computing micro datacenter based dynamic resource estimation and pricing model for IoT. In: 2015 IEEE 29th international conference on advanced information networking and applications. IEEE, pp 687–694

  39. Giang NK, Blackstock M, Lea R, Leung VC (2015) Developing IoT applications in the fog: a distributed dataflow approach. In: 2015 5th International conference on the Internet of Things (IOT). IEEE, pp 155–162

  40. ‏Luan TH, Gao L, Li Z, Xiang Y, Wei G, Sun L (2015) Fog computing: focusing on mobile users at the edge. arXiv preprint arXiv:1502.01815

  41. ‏Dastjerdi AV, Gupta H, Calheiros RN, Ghosh SK, Buyya R (2016) Fog computing: principles, architectures, and applications. In: Internet of things. Morgan Kaufmann, , pp 61–75

  42. Taneja M, Davy A (2016) Resource aware placement of data analytics platform in fog computing. Procedia Comput Sci 97:153–156

    Article  Google Scholar 

  43. Sarkar S, Misra S (2016) Theoretical modelling of fog computing: a green computing paradigm to support IoT applications. IET Netw 5(2):23–29

    Article  Google Scholar 

  44. Munir A, Kansakar P, Khan SU (2017) IFCIoT: Integrated Fog Cloud IoT: A novel architectural paradigm for the future Internet of Things. IEEE Consumer Electron Mag 6(3):74–82

    Article  Google Scholar 

  45. Kunal S, Saha A, Amin R (2019) An overview of cloud-fog computing: architectures, applications with security challenges. Security Privacy 2(4):e72

    Article  Google Scholar 

  46. Hernández-Nieves E, Hernández G, Gil-González AB, Rodríguez-González S, Corchado JM (2020) Fog computing architecture for personalized recommendation of banking products. Expert Syst Appl 140:112900

    Article  Google Scholar 

  47. Ngabo D, Wang D, Iwendi C, Anajemba JH, Ajao LA, Biamba C (2021) Blockchain-based security mechanism for the medical data at fog computing architecture of internet of things. Electronics 10(17):2110

    Article  Google Scholar 

  48. Quy VK, Hau NV, Anh DV, Ngoc LA (2022) Smart healthcare IoT applications based on fog computing: architecture, applications and challenges. Complex Intell Syst 8(5):3805–3815

    Article  Google Scholar 

  49. Natesha BV, Guddeti RMR (2022) Meta-heuristic based hybrid service placement strategies for two-level fog computing architecture. J Netw Syst Manage 30(3):47

    Article  Google Scholar 

  50. Zhang L, Ma C, Liu J, Gui H, Wang S (2023) Implementation of precision machine tool thermal error compensation in edge-cloud-fog computing architecture. J Manuf Sci Eng 145(7):071004

    Article  Google Scholar 

  51. Qin B (2023) Research on a fog computing architecture and BP algorithm application for medical big data. Intell Autom Soft Comput 37(1)

  52. Faraji F, Javadpour A, Sangaiah AK et al (2023) A solution for resource allocation through complex systems in fog computing for the internet of things. Computing. https://doi.org/10.1007/s00607-023-01199-1

Download references

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Neyshabur Branch, Islamic Azad University, Neyshabur, Iran

    Seyed Danial Alizadeh Javaheri & Hossein Monshizadeh Naeen

  2. Department of Computer Engineering, Quchan Branch, Islamic Azad University, Quchan, Iran

    Reza Ghaemi

Authors
  1. Seyed Danial Alizadeh Javaheri
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Reza Ghaemi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hossein Monshizadeh Naeen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Reza Ghaemi.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Alizadeh Javaheri, S.D., Ghaemi, R. & Monshizadeh Naeen, H. An autonomous architecture based on reinforcement deep neural network for resource allocation in cloud computing. Computing 106, 371–403 (2024). https://doi.org/10.1007/s00607-023-01220-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00607-023-01220-7

Keywords

Mathematics Subject Classification

Search

Navigation