Skip to main content

Advertisement

SpringerLink
Log in

Resource allocation for UAV-assisted 5G mMTC slicing networks using deep reinforcement learning

  • Published:
Telecommunication Systems Aims and scope Submit manuscript

Abstract

The Internet of Things (IoT) application scenarios is becoming extensive due to the quick evolution of smart devices with fifth-generation (5G) network slicing technologies. Hence, IoTs are becoming significantly important in 5G/6G networks. However, communication with IoT devices is more sensitive in disasters because the network depends on the main power supply and devices are fragile. In this paper, we consider Unmanned Aerial Vehicles (UAV) as a flying base station (BS) for the emergency communication system with 5G mMTC Network Slicing to improve the quality of user experience. The UAV-assisted mMTC creates a base station selection method to maximize the system energy efficiency. Then, the system model is reduced to the stochastic optimization-based problem using Markov Decision Process (MDP) theory. We propose a reinforcement learning-based dueling-deep-Q-networks (DDQN) technique to maximise energy efficiency and resource allocation. We compare the proposed model with DQN and Q-Learning models and found that the proposed DDQN-based model performs better for resource allocation in terms of low transmission power and maximum energy efficiency.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Data availability

All the used codes/data in this research will be available from the corresponding author on reasonable request.

References

  1. Navarro-Ortiz, J., Romero-Diaz, P., Sendra, S., Ameigeiras, P., Ramos-Munoz, J. J., & Lopez-Soler, J. M. (2020). A survey on 5G usage scenarios and traffic models. IEEE Communications Surveys Tutorials, 22(2), 905–929.

    Article  Google Scholar 

  2. Rafique, W., Qi, L., Yaqoob, I., Imran, M., Rasool, R. U., & Dou, W. (2020). Complementing IoT services through software defined networking and edge computing: A comprehensive survey. IEEE Communications Surveys Tutorials, 22(3), 1761–1804.

    Article  Google Scholar 

  3. Gupta, R. K., and Sahoo, B. (2018). Security Issues in Software-Defined Networks (2018). The IUP Journal of Information Technology, Vol. XIV, No. 2, June 2018, pp. 72–82, Available at SSRN: https://ssrn.com/abstract=3275880)

  4. Bonati, L., Polese, M., D’Oro, S., Basagni, S., & Melodia, T. (2020). Open, programmable, and virtualized 5G networks: State-of-the-art and the road ahead. Computer Networks, 182, 107516.

    Article  Google Scholar 

  5. Liu, X., Chen, M., Liu, Y., Chen, Y., Cui, S., & Hanzo, L. (2020). Artificial intelligence aided next-generation networks relying on UAVs. IEEE Wireless Communications, 28(1), 120–127.

    Article  Google Scholar 

  6. Chen, S., Liang, Y. C., Sun, S., Kang, S., Cheng, W., & Peng, M. (2020). Vision, requirements, and technology trend of 6G: How to tackle the challenges of system coverage, capacity, user data-rate and movement speed. IEEE Wireless Communications, 27(2), 218–228.

    Article  Google Scholar 

  7. Li, B., Fei, Z., & Zhang, Y. (2018). UAV communications for 5G and beyond: Recent advances and future trends. IEEE Internet of Things Journal, 6(2), 2241–2263.

    Article  Google Scholar 

  8. Wang, B., Sun, Y., Sun, Z., Nguyen, L. D., & Duong, T. Q. (2020). UAV-assisted emergency communications in social IoT: A dynamic hypergraph coloring approach. IEEE Internet of Things Journal, 7(8), 7663–7677.

    Article  Google Scholar 

  9. Khawaja, W., Guvenc, I., Matolak, D. W., Fiebig, U. C., & Schneckenburger, N. (2019). A survey of air-to-ground propagation channel modeling for unmanned aerial vehicles. IEEE Communications Surveys Tutorials, 21(3), 2361–2391.

    Article  Google Scholar 

  10. Yan, K., Ma, L., & Zhang, Y. (2020). Research on the application of 5G technology in UAV data link. In IEEE 9th joint international information technology and artificial intelligence conference (ITAIC) (Vol. 9, pp. 1115–1118). IEEE.

  11. Li, B., Fei, Z., & Zhang, Y. (2018). UAV communications for 5G and beyond: Recent advances and future trends. IEEE Internet of Things Journal, 6(2), 2241–2263.

    Article  Google Scholar 

  12. Mozaffari, M., Saad, W., Bennis, M., Nam, Y. H., & Debbah, M. (2019). A tutorial on UAVs for wireless networks: Applications, challenges, and open problems. IEEE Communications Surveys Tutorials, 21(3), 2334–2360.

    Article  Google Scholar 

  13. Xilouris, G. K., Batistatos, M. C., Athanasiadou, G. E., Tsoulos, G., Pervaiz, H. B., & Zarakovitis, C. C. (2018). UAV-assisted 5G network architecture with slicing and virtualization. In IEEE Globecom Workshops (GC Wkshps) (pp. 1–7). IEEE.

  14. Li, Y., Zhang, H., Long, K., Jiang, C., & Guizani, M. (2021). Joint resource allocation and trajectory optimization with QoS in UAV-based NOMA wireless networks. IEEE Transactions on Wireless Communications, 20, 6343–6355.

    Article  Google Scholar 

  15. Volk, M., & Sterle, J. (2021). 5G experimentation for public safety: Technologies, facilities and use cases. IEEE Access, 9, 41184–41217.

    Article  Google Scholar 

  16. Shu, Z., Taleb, T., & Song, J. (2021). Resource allocation modeling for fine-granular network slicing in beyond 5G systems. IEICE Transactions on Communications, 105, 349–363.

    Google Scholar 

  17. Gupta, R. K., & Misra, R. (2019). Machine learning-based slice allocation algorithms in 5G networks. In International conference on advances in computing, communication and control (ICAC3) (pp. 1–4). IEEE.

  18. Guo, H., Liu, J., & Zhang, J. (2018). Computation offloading for multi-access mobile edge computing in ultra-dense networks. IEEE Communications Magazine, 56(8), 14–19.

    Article  Google Scholar 

  19. Ullah, Z., Al-Turjman, F., Moatasim, U., Mostarda, L., & Gagliardi, R. (2020). UAVs joint optimization problems and machine learning to improve the 5G and Beyond communication. Computer Networks, 182, 107478.

    Article  Google Scholar 

  20. Song, J., Song, Q., Wang, Y., & Lin, P. (2021). Energy-delay tradeoff in adaptive cooperative caching for energy-harvesting ultradense networks. IEEE Transactions on Computational Social Systems, 9, 218–229.

    Article  Google Scholar 

  21. Shi, Z., Liu, J., Zhang, S., & Kato, N. (2021). Multi-agent deep reinforcement learning for massive access in 5g and beyond ultra-dense NOMA system. IEEE Transactions on Wireless Communications, 21, 3057–3070.

    Article  Google Scholar 

  22. Chen, Y., Zhang, N., Zhang, Y., Chen, X., Wu, W., & Shen, X. S. (2019). TOFFEE: Task offloading and frequency scaling for energy efficiency of mobile devices in mobile edge computing. IEEE Transactions on Cloud Computing, 9, 1634–1644.

    Article  Google Scholar 

  23. Faraci, G., Grasso, C., & Schembra, G. (2019). Reinforcement-learning for management of a 5G network slice extension with UAVs. In IEEE INFOCOM 2019-IEEE conference on computer communications workshops (INFOCOM WKSHPS) (pp. 732–737). IEEE.

  24. Faraci, G., Grasso, C., & Schembra, G. (2020). Design of a 5G network slice extension with MEC UAVs managed with reinforcement learning. IEEE Journal on Selected Areas in Communications, 38(10), 2356–2371.

    Article  Google Scholar 

  25. Liu, Y., Xiong, K., Ni, Q., Fan, P., & Letaief, K. B. (2019). UAV-assisted wireless powered cooperative mobile edge computing: Joint offloading, CPU control, and trajectory optimization. IEEE Internet of Things Journal, 7(4), 2777–2790.

    Article  Google Scholar 

  26. Wang, H., Zhao, H., Wu, W., Xiong, J., Ma, D., & Wei, J. (2019). Deployment algorithms of flying base stations: 5G and beyond with UAVs. IEEE Internet of Things Journal, 6(6), 10009–10027.

    Article  Google Scholar 

  27. Yan, S., Peng, M., & Cao, X. (2018). A game theory approach for joint access selection and resource allocation in UAV assisted IoT communication networks. IEEE Internet of Things Journal, 6(2), 1663–1674.

    Article  Google Scholar 

  28. Wang, W., Tang, J., Zhao, N., Liu, X., Zhang, X. Y., Chen, Y., & Qian, Y. (2020). Joint precoding optimization for secure SWIPT in UAV-aided NOMA networks. IEEE Transactions on Communications, 68(8), 5028–5040.

    Article  Google Scholar 

  29. Wang, J., Liu, M., Sun, J., Gui, G., Gacanin, H., Sari, H., & Adachi, F. (2020). Multiple unmanned-aerial-vehicles deployment and user pairing for nonorthogonal multiple access schemes. IEEE Internet of Things Journal, 8(3), 1883–1895.

    Article  Google Scholar 

  30. Zeng, F., Hu, Z., Xiao, Z., Jiang, H., Zhou, S., Liu, W., & Liu, D. (2020). Resource allocation and trajectory optimization for QoE provisioning in energy-efficient UAV-enabled wireless networks. IEEE Transactions on Vehicular Technology, 69(7), 7634–7647.

    Article  Google Scholar 

  31. Zhang, L., & Ansari, N. (2020). Latency-aware IoT service provisioning in UAV-aided mobile-edge computing networks. IEEE Internet of Things Journal, 7(10), 10573–10580.

    Article  Google Scholar 

  32. Gupta, R. K., Ranjan, A., Moid, M. A., & Misra, R. (2020). Deep-learning based mobile-traffic forecasting for resource utilization in 5G network slicing. In International conference on Internet of Things and connected technologies (pp. 410–424). Springer.

  33. Xie, L., Xu, J., & Zhang, R. (2018). Throughput maximization for UAV-enabled wireless powered communication networks. IEEE Internet of Things Journal, 6(2), 1690–1703.

    Article  Google Scholar 

  34. Gupta, R. K., Choubey, A., Jain, S., Greeshma, R. R., & Misra, R. (2020). Machine learning based network slicing and resource allocation for electric vehicles (EVs). In International conference on internet of things and connected technologies (pp. 333–347). Springer.

  35. Liu, X., Wang, J., Zhao, N., Chen, Y., Zhang, S., Ding, Z., & Yu, F. R. (2019). Placement and power allocation for NOMA-UAV networks. IEEE Wireless Communications Letters, 8(3), 965–968.

    Article  Google Scholar 

  36. Cai, Y., Cui, F., Shi, Q., Zhao, M., & Li, G. Y. (2018). Dual-UAV-enabled secure communications: Joint trajectory design and user scheduling. IEEE Journal on Selected Areas in Communications, 36(9), 1972–1985.

    Article  Google Scholar 

  37. Wu, Y., Yang, W., Guan, X., & Wu, Q. (2020). Energy-efficient trajectory design for UAV-enabled communication under malicious jamming. IEEE Wireless Communications Letters, 10(2), 206–210.

    Article  Google Scholar 

  38. Li, J., Zhao, H., Wang, H., Gu, F., Wei, J., Yin, H., & Ren, B. (2019). Joint optimization on trajectory, altitude, velocity, and link scheduling for minimum mission time in UAV-aided data collection. IEEE Internet of Things Journal, 7(2), 1464–1475.

  39. You, C., & Zhang, R. (2020). Hybrid offline-online design for UAV-enabled data harvesting in probabilistic LoS channels. IEEE Transactions on Wireless Communications, 19(6), 3753–3768.

    Article  Google Scholar 

  40. Ali, M. A., & Jamalipour, A. (2020). UAV placement and power allocation in uplink and downlink operations of cellular network. IEEE Transactions on Communications, 68(7), 4383–4393.

    Article  Google Scholar 

  41. Zhang, S., & Ansari, N. (2020). 3D drone base station placement and resource allocation with FSO-based backhaul in hotspots. IEEE Transactions on Vehicular Technology, 69(3), 3322–3329.

  42. Qiu, C., Wei, Z., Yuan, X., Feng, Z., & Zhang, P. (2020). Multiple UAV-mounted base station placement and user association with joint fronthaul and backhaul optimization. IEEE Transactions on Communications, 68(9), 5864–5877.

    Article  Google Scholar 

  43. Xue, Z., Wang, J., Ding, G., Wu, Q., Lin, Y., & Tsiftsis, T. A. (2018). Device-to-device communications underlying UAV-supported social networking. IEEE Access, 6, 34488–34502.

    Article  Google Scholar 

  44. Liu, M., Yang, J., & Gui, G. (2019). DSF-NOMA: UAV-assisted emergency communication technology in a heterogeneous Internet of Things. IEEE Internet of Things Journal, 6(3), 5508–5519.

    Article  Google Scholar 

  45. Chen, X., Liu, X., Chen, Y., Jiao, L., & Min, G. (2021). Deep Q-network based resource allocation for UAV-assisted ultra-dense networks. Computer Networks, 196, 108249.

    Article  Google Scholar 

  46. Yan, Z., Zhang, Z., & Meng, Y. (2021). Energy efficiency optimization for UAV-assisted mMTC networks with altitude differences. In IEEE/CIC international conference on communications in China (ICCC) (pp. 306–311). IEEE.

  47. Al-Hourani, A., Kandeepan, S., & Lardner, S. (2014). Optimal LAP altitude for maximum coverage. IEEE Wireless Communications Letters, 3(6), 569–572.

    Article  Google Scholar 

  48. Mozaffari, M., Saad, W., Bennis, M., & Debbah, M. (2015). Drone small cells in the clouds: Design, deployment and performance analysis. In IEEE global communications conference (GLOBECOM) (pp. 1–6). IEEE.

  49. Wang, Z., Schaul, T., Hessel, M., Hasselt, H., Lanctot, M., & Freitas, N. (2016). Dueling network architectures for deep reinforcement learning. In International conference on machine learning (pp. 1995–2003). PMLR.

Download references

Funding

The preparation of the manuscript is not funded.

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Indian Institute of Technology Patna, Patna, Bihar, 801106, India

    Rohit Kumar Gupta, Saubhik Kumar & Rajiv Misra

Authors
  1. Rohit Kumar Gupta
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Saubhik Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rajiv Misra
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

The study, implementation and experiments were done by all of the authors. RKG prepared the materials, ran the simulations, and conducted the analysis, and SK helped in the simulations under the supervision of Dr. RM. RKG wrote the first draft of the manuscript, and all contributors provided feedback. The final manuscript was read and approved by all of the authors.

Corresponding author

Correspondence to Rohit Kumar Gupta.

Ethics declarations

Conflict of interest

There are no conflict of interest declared by the authors.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gupta, R.K., Kumar, S. & Misra, R. Resource allocation for UAV-assisted 5G mMTC slicing networks using deep reinforcement learning. Telecommun Syst 82, 141–159 (2023). https://doi.org/10.1007/s11235-022-00974-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11235-022-00974-3

Keywords

Search

Navigation