Skip to main content
SpringerLink
Log in

Intelligent Resource Scheduling

  • Chapter
  • First Online:
Intelligent Internet of Things Networks

Part of the book series: Wireless Networks ((WN))

  • 231 Accesses

Abstract

The continued growth in the number and applications of Internet of Things (IoT) connected devices makes it more challenging to meet multi-dimensional QoS within the same IoT network. In this chapter, we first design a network slicing architecture over the SDN-based long-range wide area network. The SDN controller can dynamically split the network into multiple virtual networks according to different business requirements. Then, a Continuous-Decision virtual network embedding scheme relying on Reinforcement Learning (CDRL) is proposed, two traditional heuristic embedding algorithms as well as the classic reinforcement learning aided embedding algorithm are used for benchmarking our proposed CDRL algorithm. Finally, we propose a hybrid intelligent control architecture, which adopts the centralized training and distributed execution paradigm. A centralized critic is introduced to ease the training process of the distributed network nodes. Besides, considering the competitive behavior of users, we formulate the resource allocation problem as a multi-user competition game model. Based on this, we proposed a multi-agent reinforcement learning-based SFCs deployment algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    The seq2seq is a common model in the field of Natural Language processing (NLP), which is a widely used architecture for machine translation and summarization relying on a recurrent neural network as one of its building blocks [38, 39].

References

  1. T. Mai, H. Yao, N. Zhang, W. He, D. Guo, M. Guizani, Transfer reinforcement learning aided distributed network slicing optimization in industrial IoT. IEEE Trans. Ind. Inform. 18(6), 4308–4316 (2021)

    Article  Google Scholar 

  2. H. Yao, S. Ma, J. Wang, P. Zhang, C. Jiang, S. Guo, A continuous-decision virtual network embedding scheme relying on reinforcement learning. IEEE Trans. Netw. Service Manag. 17(2), 864–875 (2020)

    Article  Google Scholar 

  3. Y. Zhu, H. Yao, T. Mai, W. He, N. Zhang, M. Guizani, Multi-agent reinforcement learning aided service function chain deployment for Internet of Things. IEEE Internet Things J. 9(17), 15674–15684 (2022)

    Article  Google Scholar 

  4. G. Han, J. Tu, L. Liu, M. Martinez-Garcia, C. Choi, An intelligent signal processing data denoising method for control systems protection in the industrial Internet of Things. IEEE Trans. Ind. Inform. 18(4), 2684–2692 (2021)

    Article  Google Scholar 

  5. A. Lavric, V. Popa, Internet of things and LoRa low-power wide-area networks: a survey, in 2017 International Symposium on Signals, Circuits and Systems (ISSCS) (IEEE, 2017), pp. 1–5

    Google Scholar 

  6. G. Han, J. Tu, L. Liu, M. Martínez-García, Y. Peng, Anomaly detection based on multidimensional data processing for protecting vital devices in 6g-enabled massive IIoT. IEEE Internet Things J. 8(7), 5219–5229 (2021)

    Article  Google Scholar 

  7. S. Wijethilaka, M. Liyanage, Survey on network slicing for internet of things realization in 5g networks. IEEE Commun. Surveys Tutorials 23(2), 957–994 (2021)

    Article  Google Scholar 

  8. C. Qiu, H. Yao, F.R. Yu, F. Xu, C. Zhao, Deep q-learning aided networking, caching, and computing resources allocation in software-defined satellite-terrestrial networks. IEEE Trans. Veh. Technol. 68(6), 5871–5883 (2019)

    Article  Google Scholar 

  9. M. Bor, J.E. Vidler, U. Roedig, Lora for the Internet of Things (2016)

    Google Scholar 

  10. J. Wang, C. Jiang, H. Zhang, Y. Ren, K.-C. Chen, L. Hanzo, Thirty years of machine learning: the road to pareto-optimal wireless networks. IEEE Commun. Surv. Tutorials 22(3), 1472–1514 (2020)

    Article  Google Scholar 

  11. S.J. Pan, Q. Yang, A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22(10), 1345–1359 (2009)

    Article  Google Scholar 

  12. B. K. Al-Shammari, N. Al-Aboody, H. S. Al-Raweshidy, IoT traffic management and integration in the QoS supported network. IEEE Internet Things J. 5(1), 352–370 (2017)

    Article  Google Scholar 

  13. K. Benzekki, A. El Fergougui, A. Elbelrhiti Elalaoui, Software-defined networking (SDN): a survey. Security Communication Networks 9(18), 5803–5833 (2016)

    Article  Google Scholar 

  14. C. Qiu, H. Yao, C. Jiang, S. Guo, F. Xu, Cloud computing assisted blockchain-enabled Internet of Things. IEEE Trans. Cloud Comput. 10(1), 247–257. (2019)

    Article  Google Scholar 

  15. O. Georgiou, U. Raza, Low power wide area network analysis: Can LoRa scale? IEEE Wirel. Commun. Lett. 6(2), 162–165 (2017)

    Article  Google Scholar 

  16. S. Dawaliby, A. Bradai, Y. Pousset, Distributed network slicing in large scale IoT based on coalitional multi-game theory. IEEE Trans. Netw. Service Manag. 16(4), 1567–1580 (2019)

    Article  Google Scholar 

  17. K. Xue, B. Zhu, Q. Yang, N. Gai, D. S. Wei, N. Yu, InPPTD: a lightweight incentive-based privacy-preserving truth discovery for crowdsensing systems. IEEE Internet Things J. 8(6), 4305–4316 (2020)

    Article  Google Scholar 

  18. T. Mai, H. Yao, N. Zhang, L. Xu, M. Guizani, S. Guo, Cloud mining pool aided blockchain-enabled internet of things: An evolutionary game approach. IEEE Trans. Cloud Comput. (2021)

    Google Scholar 

  19. Y. He, G. Han, J. Jiang, H. Wang, M. Martinez-Garcia, A trust update mechanism based on reinforcement learning in underwater acoustic sensor networks. IEEE Trans. Mobile Comput. 21(3), 811–821 (2020)

    Article  Google Scholar 

  20. B. Eysenbach, R. Salakhutdinov, S. Levine, Search on the replay buffer: bridging planning and reinforcement learning (2019). arXiv preprint arXiv:1906.05253

    Google Scholar 

  21. M.E. Taylor, P. Stone, Transfer learning for reinforcement learning domains: a survey. J. Mach. Learn. Res. 10(7) (2009)

    Google Scholar 

  22. A. Fischer, J.F. Botero, M.T. Beck, H. De Meer, X. Hesselbach, Virtual network embedding: a survey. IEEE Commun. Surv. Tutorials 15(4), 1888–1906 (2013)

    Article  Google Scholar 

  23. N.M.M.K. Chowdhury, R. Boutaba, A survey of network virtualization. Comput. Netw. 54(5), 862–876 (2010)

    Article  MATH  Google Scholar 

  24. D. Drutskoy, E. Keller, J. Rexford, Scalable network virtualization in software-defined networks. IEEE Internet Comput. 17(2), 20–27 (2013)

    Article  Google Scholar 

  25. P. Zhang, X. Pang, Y. Bi, H. Yao, H. Pan, N. Kumar, DSCD: delay sensitive cross-domain virtual network embedding algorithm. IEEE Trans. Netw. Sci. Eng. 7(4), 2913–2925 (2020)

    Article  Google Scholar 

  26. Y. Zeng, R. Zhang, Efficient Mapping of Virtual Networks onto a Shared Substrate (Washington University in St Louis, 2006)

    Google Scholar 

  27. Y. Zhu, M.H. Ammar, Algorithms for assigning substrate network resources to virtual network components, in 25th IEEE International Conference on Computer Communications, Barcelona, Spain, 2006, pp. 23–29

    Google Scholar 

  28. Z. Liu, M. Wu, Exact solutions of VNE: a survey. China Commun. 13(6), 48–62 (2016)

    Article  Google Scholar 

  29. M. Yu, Y. Yi, J. Rexford, M. Chiang, Rethinking virtual network embedding: substrate support for path splitting and migration. Comput. Commun. Rev. 38(2), 17–29 (2008)

    Article  Google Scholar 

  30. A. Razzaq, M.S. Rathore, An approach towards resource efficient virtual network embedding, in International Conference on Evolving Internet, Valencia, Spain, 2010, pp. 68–73

    Google Scholar 

  31. X. Cheng, S. Su, Z. Zhang, H. Wang, F. Yang, Y. Luo, J. Wang, Virtual network embedding through topology-aware node ranking. Comput. Commun. Rev. 41(2), 38–47 (2011)

    Article  Google Scholar 

  32. X. Hesselbach, J.R. Amazonas, S. Villanueva, J.F. Botero, Coordinated node and link mapping VNE using a new paths algebra strategy. J. Netw. Comput. Appl. 69, 14–26 (2016)

    Article  Google Scholar 

  33. J. Wang, C. Jiang, H. Zhang, Y. Ren, K.-C. Chen, L. Hanzo, Thirty years of machine learning: the road to pareto-optimal wireless networks. IEEE Commun. Surv. Tutorials (2020) https://doi.org/10.1109/COMST.2020.2965856

  34. D.E. Goldberg, Genetic algorithms in search, optimization, and machine learning, in Ethnographic Praxis in Industry Conference, Portland, US, 1988, pp. 3104–3112

    Google Scholar 

  35. L.P. Kaelbling, M.L. Littman, A.W. Moore, Reinforcement learning: a survey. J. Artif. Intell. Res. 4(1), 237–285 (1996)

    Article  Google Scholar 

  36. R.S. Sutton, A.G. Barto, Reinforcement learning: an introduction. IEEE Trans. Neural Netw. 9(5), 1054–1054 (1998)

    Article  Google Scholar 

  37. Y. Lecun, Y. Bengio, G.E. Hinton, Deep learning. Nature 521(7553), 436–444 (2015)

    Article  Google Scholar 

  38. K. Cho, B. Van Merrienboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, Y. Bengio, Learning phrase representations using RNN encoder–decoder for statistical machine translation, in Conference on Empirical Methods in Natural Language Processing, Doha, Qatar, 2014, pp. 1724–1734

    Google Scholar 

  39. I. Sutskever, O. Vinyals, Q. V. Le, Sequence to sequence learning with neural networks, in Annual Conference on Neural Information Processing Systems, Montreal, Canada, 2014, pp. 3104–3112

    Google Scholar 

  40. H. Yao, C. Xu, M. Li, P. Zhang, L. Wang, A novel reinforcement learning algorithm for virtual network embedding. Neurocomputing 284, 1–9 (2018)

    Article  Google Scholar 

  41. H. Yao, B. Zhang, L. Maozhen, P. Zhang, L. Wang, RDAM: a reinforcement learning based dynamic attribute matrix representation for virtual network embedding. IEEE Trans. Emer. Topics Comput. PP(99), 1–1 (2019)

    Google Scholar 

  42. R. Sutton, A. Barto, Reinforcement Learning: An Introduction, 2nd edn. a Bradford book (2018)

    Google Scholar 

  43. C. Watkins, P. Dayan, Q-learning[J]. Mach. Learn. 8(3), 279–292 (1992)

    Article  MATH  Google Scholar 

  44. R.J. Williams, Simple statistical gradient-following algorithms for connectionist reinforcement learning. Mach. Learn. 8(3–4), 229–256 (1992)

    Article  MATH  Google Scholar 

  45. D. Silver, G. Lever, N. Heess, T. Degris, D. Wierstra, M. Riedmiller, Deterministic policy gradient algorithms, in 31st International Conference on Machine Learning, ICML 2014, vol. 1 (2014)

    Google Scholar 

  46. S. Hougardy, The Floyd-Warshall algorithm on graphs with negative cycles. Inform. Process. Lett. 110(8), 279–281 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  47. P. Koehn, Pharaoh: a beam search decoder for phrase-based statistical machine translation models. (2004), pp. 115–124

    Google Scholar 

  48. E.Z.M. Thomas, Generation and analysis of random graphs to model internetworks. College Comput. Georgia Institute Technol. 63(4), 413–442 (1994)

    Google Scholar 

  49. M. Abadi, P. Barham, J. Chen, Z. Chen, A. Davis, J. Dean, M. Devin, S. Ghemawat, G. Irving, M. Isard, TensorFlow: a system for large-scale machine learning, in 25th IEEE International Conference on Computer Communications, Georgia, USA, 2016, pp. 265–283

    Google Scholar 

  50. Wilcoxon, F., Individual comparisons of grouped data by ranking methods. J. Econ. Entomol. 39(2), 269–270

    Google Scholar 

  51. R. Duan, J. Wang, C. Jiang, H. Yao, Y. Ren, Y. Qian, Resource allocation for multi-UAV aided IoT NOMA uplink transmission systems. IEEE Internet Things J. 6(4), 7025–7037 (2019)

    Article  Google Scholar 

  52. L. Cui, F.P. Tso, S. Guo, W. Jia, K. Wei, W. Zhao, Enabling heterogeneous network function chaining. IEEE Trans. Parallel Distrib. Syst. 30(4), 842–854 (2019)

    Article  Google Scholar 

  53. L. Qu, C. Assi, M.J. Khabbaz, Y. Ye, Reliability-aware service function chaining with function decomposition and multipath routing. IEEE Trans. Netw. Serv. Manag. 17(2), 835–848 (2020)

    Article  Google Scholar 

  54. S. Da̧ŕOro, L. Galluccio, S. Palazzo, G. Schembra, Exploiting congestion games to achieve distributed service chaining in NFV networks. IEEE J. Sel. Areas Commun. 35(2), 407–420 (2017)

    Google Scholar 

  55. J. Liu, W. Lu, F. Zhou, P. Lu, Z. Zhu, On dynamic service function chain deployment and readjustment. IEEE Trans. Netw. Serv. Manag. 14(3), 543–553 (2017)

    Article  Google Scholar 

  56. H. Hawilo, M. Jammal, A. Shami, Network function virtualization-aware orchestrator for service function chaining placement in the cloud. IEEE J. Sel. Areas Commun. 37(3), 643–655 (2019)

    Article  Google Scholar 

  57. A.M. Medhat, T. Taleb, A. Elmangoush, G.A. Carella, S. Covaci, T. Magedanz, Service function chaining in next generation networks: state of the art and research challenges. IEEE Commun. Mag. 55(2), 216–223 (2017)

    Article  Google Scholar 

  58. J. Pei, P. Hong, K. Xue, D. Li, Efficiently embedding service function chains with dynamic virtual network function placement in geo-distributed cloud system. IEEE Trans. Parallel Distrib. Syst. 30(10), 2179–2192 (2019)

    Article  Google Scholar 

  59. S. Bian, X. Huang, Z. Shao, X. Gao, Y. Yang, Service chain composition with resource failures in NFV systems: a game-theoretic perspective. IEEE Trans. Netw. Serv. Manag. 18(1), 224–239 (2021)

    Article  Google Scholar 

  60. J. Wang, H. Qi, K. Li, X. Zhou, PRSFC-IoT: a performance and resource aware orchestration system of service function chaining for Internet of Things. IEEE Internet Things J. 5(3), 1400–1410 (2018)

    Article  Google Scholar 

  61. W. Ren, Y. Sun, H. Luo, M.S. Obaidat, A new scheme for IoT service function chains orchestration in SDN-IoT network systems. IEEE Syst. J. 13(4), 4081–4092 (2019)

    Article  Google Scholar 

  62. T. Mai, H. Yao, N. Zhang, W. He, D. Guo, M. Guizani, Transfer reinforcement learning aided distributed network slicing resource optimization in industrial IoT. IEEE Trans. Ind. Inform. 18(6), 4308–4316 (2021)

    Article  Google Scholar 

  63. Y. He, F. R. Yu, N. Zhao, V.C.M. Leung, H. Yin, Software-defined networks with mobile edge computing and caching for smart cities: a big data deep reinforcement learning approach. IEEE Commun. Mag. 55(12), 31–37 (2017)

    Article  Google Scholar 

  64. L. Zhao, J. Wang, J. Liu, N. Kato, Routing for crowd management in smart cities: a deep reinforcement learning perspective. IEEE Commun. Mag. 57(4), 88–93 (2019)

    Article  Google Scholar 

  65. C. Qiu, H. Yao, C. Jiang, S. Guo, F. Xu, Cloud computing assisted blockchain-enabled internet of things. IEEE Trans. Cloud Comput. 10(1), 247–257 (2019)

    Article  Google Scholar 

  66. L. Gu, D. Zeng, W. Li, S. Guo, A.Y. Zomaya, H. Jin, Intelligent VNF orchestration and flow scheduling via model-assisted deep reinforcement learning. IEEE J. Sel. Areas Commun. 38(2), 279–291 (2020)

    Article  Google Scholar 

  67. T.A.Q. Pham, J.-M. Sanner, C. Morin, Y. Hadjadj-Aoul, Virtual network function–forwarding graph embedding: a genetic algorithm approach. Int. J. Commun. Syst. 33(10), e4098 (2020)

    Google Scholar 

  68. A.S. Kumar, L. Zhao, X. Fernando, Mobility aware channel allocation for 5g vehicular networks using multi-agent reinforcement learning, in ICC 2021—IEEE International Conference on Communications (2021), pp. 1–6

    Google Scholar 

  69. Y. Xiao, Q. Zhang, F. Liu, J. Wang, M. Zhao, Z. Zhang, J. Zhang, NFVdeep: adaptive online service function chain deployment with deep reinforcement learning, in Proceedings of the International Symposium on Quality of Service (2019), pp. 1–10

    Google Scholar 

  70. W. Lu, Z. Zhu, Dynamic service provisioning of advance reservation requests in elastic optical networks. J. Lightwave Technol. 31(10), 1621–1627 (2013)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Beijing University of Posts and Telecommunications, Beijing, China

    Haipeng Yao

  2. Mohamed bin Zayed University of Artificial Intelligence (MBZUAI), Abu Dhabi, United Arab Emirates

    Mohsen Guizani

Authors
  1. Haipeng Yao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohsen Guizani
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Yao, H., Guizani, M. (2023). Intelligent Resource Scheduling. In: Intelligent Internet of Things Networks . Wireless Networks. Springer, Cham. https://doi.org/10.1007/978-3-031-26987-5_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-26987-5_5

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-26986-8

  • Online ISBN: 978-3-031-26987-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation