Skip to main content
SpringerLink
Log in

Reinforcement R-learning model for time scheduling of on-demand fog placement

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

On the fly deployment of fog nodes near users provides the flexibility of pushing services anywhere and whenever needed. Nevertheless, taking a real-life scenario, the cloud might limit the number of fogs to place for minimizing the complexity of monitoring a large number of fogs and cost for volunteers that do not offer their resources for free. This implies choosing the right time and best volunteer to create a fog which the cloud can benefit from is essential. This choice is subject to study the demand of a particular location for services in order to maximize the resources utilization of these fogs. A simple algorithm will not be able to explore randomly changing users’ demands. Therefore, there is a need for an intelligent model capable of scheduling fog placement based on the user’s requests. In this paper, we propose a Fog Scheduling Decision model based on reinforcement R-learning, which focuses on studying the behavior of service requesters and produces a suitable fog placement schedule based on the concept of average reward. Our model aims to decrease the cloud’s load by utilizing the maximum available fogs resources over different locations. An implementation of our proposed R-learning model is provided in the paper, followed by a series of experiments on a real dataset to prove its efficiency in utilizing fog resources and minimizing the cloud’s load. We also demonstrate the ability of our model to improve over time by adapting the new demand of users. Experiments comparing the decisions of our model with two other potential fog placement approaches used for task/service scheduling (threshold based and random based) show that the number of processed requests performed by the cloud decreases from 100 to 30% with a limited number of fogs to push. These results demonstrate that our proposed Fog Scheduling Decision model plays a crucial role in the placement of the on-demand fog to the right location at the right time while taking into account the user’s needs.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Patel KK, Patel SM et al (2016) Internet of things-IOT: definition, characteristics, architecture, enabling technologies, application and future challenges. Int J Eng Sci Comput 6(5):6122–6131

    Google Scholar 

  2. Gubbi J, Buyya R, Marusic S, Palaniswami M (2013) Internet of things (IOT): a vision, architectural elements, and future directions. Future Gener Comput Syst 29:1645–1660

    Article  Google Scholar 

  3. Alberti AM, Singh D (2013) Internet of things: perspectives, challenges and opportunities. In: International Workshop on Telecommunications (IWT 2013), pp 1–6

  4. Tordera E M, Masip-Bruin X, Garcia-Alminana J, Jukan A, Ren G-J, Zhu J, Farré J (2016) What is a fog node a tutorial on current concepts towards a common definition. arXiv preprint arXiv:1611.09193

  5. Bonomi F, Milito R, Zhu J, Addepalli S (2012) Fog computing and its role in the internet of things. In: Proceedings of the First Edition of the MCC Workshop on Mobile Cloud Computing. ACM, pp 1–6

  6. Sami H, Mourad A (2018) Towards dynamic on-demand fog computing formation based on containerization technology. In: Proceedings of the 2018 International Conference on Computational Science and Computational Intelligence–CSCI’18. IEEE Computer Society

  7. Kratzke N (2017) About microservices, containers and their underestimated impact on network performance. arXiv preprint arXiv:1710.04049

  8. Sayfan G (2017) Mastering kubernetes. Packt Publishing Ltd, Birmingham

    Google Scholar 

  9. Fatema K, Emeakaroha VC, Healy PD, Morrison JP, Lynn T (2014) A survey of cloud monitoring tools: taxonomy, capabilities and objectives. J Parallel Distrib Comput 74(10):2918–2933

    Article  Google Scholar 

  10. Roman R, Lopez J, Mambo M (2018) Mobile edge computing, fog et al.: a survey and analysis of security threats and challenges. Future Gener Comput Syst 78:680–698

    Article  Google Scholar 

  11. Yi S, Li C, Li Q (2015) A survey of fog computing: concepts, applications and issues. In: Proceedings of the 2015 Workshop on Mobile Big Data. ACM, pp 37–42

  12. Gosavi A et al (2015) Simulation-based optimization. Springer, Berlin

    MATH  Google Scholar 

  13. Watkins CJ, Dayan P (1992) Q-learning. Mach Learn 8(3–4):279–292

    MATH  Google Scholar 

  14. Nasa dataset-two months of http logs from a busy www server. [Online]. https://ita.ee.lbl.gov/html/traces.html

  15. Hao Z, Novak E, Yi S, Li Q (2017) Challenges and software architecture for fog computing. IEEE Internet Comput 21(2):44–53

    Article  Google Scholar 

  16. Mahmud R, Kotagiri R, Buyya R (2018) Fog computing: a taxonomy, survey and future directions. In: Internet of everything. Springer, pp 103–130

  17. Tout H, Talhi C, Kara N, Mourad A (2017) Smart mobile computation offloading: centralized selective and multi-objective approach. Expert Syst Appl 80:1–13

    Article  Google Scholar 

  18. Marrouche W, Harmanani HM (2018) Heuristic approaches for the open-shop scheduling problem. In: Information Technology-New Generations. Springer, pp 691–699

  19. Dbouk T, Mourad A, Otrok H, Tout H, Talhi C (2019) A novel ad-hoc mobile edge cloud offering security services through intelligent resource-aware offloading. IEEE Trans Netw Serv Manag. https://doi.org/10.1109/TNSM.2019.2939221

    Article  Google Scholar 

  20. Chen M, Hao Y (2018) Task offloading for mobile edge computing in software defined ultra-dense network. IEEE J Sel Areas Commun 36(3):587–597

    Article  Google Scholar 

  21. Zhang J, Hu X, Ning Z, Ngai EC-H, Zhou L, Wei J, Cheng J, Hu B (2017) Energy-latency tradeoff for energy-aware offloading in mobile edge computing networks. IEEE Internet Things J 5(4):2633–2645

    Article  Google Scholar 

  22. Pham X-Q, Man ND, Tri NDT, Thai NQ, Huh E-N (2017) A cost-and performance-effective approach for task scheduling based on collaboration between cloud and fog computing. Int J Distrib Sens Netw 13(11):1550147717742073

    Article  Google Scholar 

  23. Pham X-Q, Huh E-N (2016) Towards task scheduling in a cloud-fog computing system. In: 18th Asia-Pacific Network Operations and Management Symposium (APNOMS). IEEE, pp 1–4

  24. Zeng D, Gu L, Guo S, Cheng Z, Yu S (2016) Joint optimization of task scheduling and image placement in fog computing supported software-defined embedded system. IEEE Trans Comput 65(12):3702–3712

    Article  MathSciNet  Google Scholar 

  25. Kherraf N, Alameddine HA, Sharafeddine S, Assi C, Ghrayeb A (2019) Optimized provisioning of edge computing resources with heterogeneous workload in iot networks. IEEE Trans Netw Serv Manag 16(5):459–474

    Article  Google Scholar 

  26. Alameddine HA, Sharafeddine S, Sebbah S, Ayoubi S, Assi C (2019) Dynamic task offloading and scheduling for low-latency iot services in multi-access edge computing. IEEE J Sel Areas Commun 37(3):668–682

    Article  Google Scholar 

  27. Yousefpour A, Ishigaki G, Jue J P (2017) Fog computing: Towards minimizing delay in the internet of things. In: 2017 IEEE International Conference on Edge Computing (EDGE). IEEE, pp 17–24

  28. Hoque S, de Brito M S, Willner A, Keil O, Magedanz T (2017) Towards container orchestration in fog computing infrastructures. In: Proceedings of the (2017) IEEE 41st Annual on Computer Software and Applications Conference (COMPSAC), vol 2. IEEE, pp 294–299

  29. Felter W, Ferreira A, Rajamony R, Rubio J (2015) An updated performance comparison of virtual machines and linux containers. In: 2015 IEEE International Symposium Performance on Analysis of Systems and Software (ISPASS). IEEE

  30. Hong H-J, Tsai P-H, Hsu C-H (2016) “Dynamic module deployment in a fog computing platform. In: 18th Asia-Pacific on Network Operations and Management Symposium (APNOMS). IEEE, pp 1–6

  31. Knox WB, Setapen AB, Stone P (2011) Reinforcement learning with human feedback in mountain car. In: 2011 AAAI Spring Symposium Series

  32. Gosavi A (2011) A tutorial for reinforcement learning. Department of Engineering Management and Systems Engineering

  33. Skarlat O, Nardelli M, Schulte S, Borkowski M, Leitner P (2017) Optimized iot service placement in the fog. Serv Oriented Comput Appl 11(4):427–443

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science & Mathematics, Lebanese American University, Beirut, Lebanon

    Peter Farhat, Hani Sami & Azzam Mourad

Authors
  1. Peter Farhat
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hani Sami
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Azzam Mourad
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Azzam Mourad.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Farhat, P., Sami, H. & Mourad, A. Reinforcement R-learning model for time scheduling of on-demand fog placement. J Supercomput 76, 388–410 (2020). https://doi.org/10.1007/s11227-019-03032-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-019-03032-z

Keywords

Search

Navigation