Skip to main content
SpringerLink
Log in

A Heuristic Deep Q Learning for Offloading in Edge Devices in 5 g Networks

  • Research
  • Published:
Journal of Grid Computing Aims and scope Submit manuscript

Abstract

The 5G Wireless Environments have huge data transmission; therefore, there is an increase in the requests for computational tasks from Intelligent Wireless Mobile Nodes. This computational capability leads to high reliability and low latency in a 5G network. Mobile edge computing (MEC) allows end systems with constrained computing capacity to handle computationally demanding tasks and offer accurate alternatives. The MEC server’s physical position is nearer to WN than other servers, which satisfies the demands for low latency and excellent dependability. To overcome the issues of existing work, such as low latency, offloading and task scheduling, the proposed method provides efficient results. In this work for job scheduling, Multi-agent Collaborative Deep Reinforcement Learning based Scheduling Algorithm with a double deep Q network (DQN) is used in the MEC system. To minimize the total Latency in Wireless Nodes, it uses Karush-Kuhn-Tucker (KKT) approach. This approach provides the optimum solutions to the partial and complete offloading tasks. The double deep Q network (DQN) reduces energy consumption and offers better convergence Between the Wireless Nodes. Compared to traditional algorithms like DeMDRL and BiDRL, the proposed MDRL-DDQN demonstrates superior performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Data Availability

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

References

  1. Ke, H., Wang, H., Sun, H.: Multi-agent deep reinforcement learning-based partial task offloading and resource allocation in edge computing environment. Electron. 11(15), 2394 (2022)

    Article  Google Scholar 

  2. Li, Y., Li, J., Pang, J.: A graph attention mechanism-based multiagent reinforcement-learning method for task scheduling in edge computing. Electron. 11(9), 1357 (2022)

    Article  Google Scholar 

  3. Li, D., Shaoyi, X., Li, P.: Deep reinforcement learning-empowered resource allocation for mobile edge computing in cellular v2x networks. Sens. 21(2), 372 (2021)

    Article  Google Scholar 

  4. Yang, W.; Wang, N.; Guan, Z.; Wu, L.; Du, X.; Guizani, M.: A Practical Cross-Device Federated Learning Framework over 5G Networks. IEEE Wirel. Commun. 29(6), 128–134 (2022)

  5. Lu, F., Zhao, H., Zhao, X., Wang, X., Saleem, A., Zheng, G.: Investigation of near-field source localization using uniform rectangular Array. Electron. 11, 1916 (2022)

    Article  Google Scholar 

  6. Lu, J., Hao, Y., Wu, K., Chen, Y., Wang, Q.: Dynamic offloading for energy-aware scheduling in a mobile cloud. J. King Saud-Univ.-Comput. Inf. Sci. 34, 3167–3177 (2022)

    Google Scholar 

  7. Guo, Y.; Li, H.: Exploration on the optimal application of Mobile cloud computing in Enterprise financial management under 5G network architecture. Adv. Multimed.7500014, (2022)

  8. uz Zaman, S.K., Jehangiri, A.I., Maqsood, T., Ahmad, Z., Umar, A.I., Shuja, J., Alanazi, E., Alasmary, W.: Mobility-aware computational offloading in mobile edge networks: a survey. Clust. Comput. 24, 2735–2756 (2021)

    Article  Google Scholar 

  9. Plachy, J., Becvar, Z., Strinati, E.C., Pietro, N.D.: Dynamic allocation of computing and communication resources in multi-access edge computing for Mobile users. IEEE Trans. Netw. Serv. Manag. 18, 2089–2106 (2021)

    Article  Google Scholar 

  10. Zhao, F., Chen, Y., Zhang, Y., Liu, Z., Chen, X.: Dynamic offloading and resource scheduling for Mobile-edge computing with energy harvesting devices. IEEE Trans. Netw. Serv. Manag. 18, 2154–2165 (2021)

    Article  Google Scholar 

  11. Shuja, J., Bilal, K., Alasmary, W., Sinky, H., Alanazi, E.: Applying machine learning techniques for caching in next-generation edge networks: a comprehensive survey. J. Netw. Comput. Appl. 181, 103005 (2021)

    Article  Google Scholar 

  12. Abbas, N., Zhang, Y., Taherkordi, A., Skeie, T.: Mobile edge computing: a survey. IEEE Internet Things J. 5, 450–465 (2018)

    Article  Google Scholar 

  13. Li, C., Wang, H., Song, R.: Mobility-aware offloading and resource allocation in NOMA-MEC systems via DC. IEEE Commun. Lett. 26, 1091–1095 (2022)

    Article  Google Scholar 

  14. Tian, K., Chai, H., Liu, Y., Liu, B.: Edge intelligence empowered dynamic offloading and resource management of MEC for Smart City internet of things. Electron. 11, 879 (2022)

    Article  Google Scholar 

  15. Chen, C., Zeng, Y., Li, H., Liu, Y., Wan, S.: A multi-hop task offloading decision model in MEC-enabled internet of vehicles. IEEE Int. Things J. (2022)

  16. Kuang, Z., Li, L., Gao, J., Zhao, L., Liu, A.: Partial offloading scheduling and power allocation for mobile edge computing systems.IEEE. Int. Things J. 6, 6774–6785 (2019)

    Article  Google Scholar 

  17. Yang, L., Yao, H., Wang, J., Jiang, C., Liu, Y.: Multi-UAV enabled load-balance Mobile edge computing for IoT networks (IEEE IoT journal). IEEE Internet Things J. 7, 6898–6908 (2020)

    Article  Google Scholar 

  18. Zhang, K., Hu, Y., Tian, F., Li, C.: A coalition-Structure’s generation method for solving cooperative computing problems in edge computing environments. Inf. Sci. 536, 372–390 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  19. Zhu, K., Zhang, T.: Deep Reinforcement Learning Based Mobile Robot Navigation: A Review. Tsinghua Sci. Technol. 26, 18 (2021)

    Article  Google Scholar 

  20. Wang, S., Sheng, H., Yang, D., Zhang, Y.,Yubin, W., Wang, S.: Extendable Multiple Nodes Recurrent Tracking Framework with RTU++. IEEE Trans. Image Process., (2022)

  21. Wu, Y., Sheng, H., Zhang, Y., Wang, S., Xiong, Z., ... Ke, W.: Hybrid motion model for multiple object tracking in Mobile devices. IEEE Internet Things J., (2022)

  22. Zhao, J., Gao, F., Jia, W., Yuan, W., Jin, W.: Integrated sensing and communications for UAV communications with jittering effect. IEEE Wirel. Commun. Lett., (2023)

  23. Xiao, Z., Shu, J., Jiang, H., Min, G., Chen, H., et al.: Perception task offloading with collaborative computation for autonomous driving. IEEE J. Select. Areas Commun. 41(2), 457–473 (2023)

    Article  Google Scholar 

  24. Dai, X., Xiao, Z., Jiang, H., Alazab, M., Lui, J.C.S., Min, G., et al.: Task offloading for cloud-assisted fog computing with dynamic service caching in Enterprise management systems. IEEE Trans. Indust. Inf. 19(1), 662–672 (2023)

    Article  Google Scholar 

  25. Tian, J., Hou, M., Bian, H., Li, J.: Variable surrogate model-based particle swarm optimization for high-dimensional expensive problems. Complex. Intell. Syst. 1–49 (2022)

  26. Li, B., Zhang, M., Rong, Y., Han, Z.: Transceiver optimization for wireless powered time-division duplex MU-MIMO systems: non-robust and robust designs. IEEE Trans. Wirel. Commun. 21(6), 4594–4607 (2021)

    Article  Google Scholar 

  27. Li, B., Li, Q., Zeng, Y., Rong, Y., Zhang, R.: 3D trajectory optimization for energy-efficient UAV communication: a control design perspective. IEEE Trans. Wirel. Commun. 21(6), 4579–4593 (2021)

    Article  Google Scholar 

  28. Zhao, Z., Xu, G., Zhang, N., Zhang, Q.: Performance analysis of the hybrid satellite-terrestrial relay network with opportunistic scheduling over generalized fading channels. IEEE Trans. Veh. Technol. 71(3), 2914–2924 (2022)

    Article  Google Scholar 

  29. Liu, G.: A Q-learning-based distributed routing protocol for frequency-switchable magnetic induction-based wireless underground sensor networks. Futur. Gener. Comput. Syst. 139, 253–266 (2023)

    Article  Google Scholar 

  30. Cao, K., Wang, B., Ding, H., Lv, L., Dong, R., Cheng, T., et al.: Improving physical layer security of uplink NOMA via energy harvesting jammers. IEEE Trans. Inf. Forensic. Secur. 16, 786–799 (2021)

    Article  Google Scholar 

  31. Cao, K., Wang, B., Ding, H., Lv, L., Tian, J., Hu, H., et al.: Achieving reliable and secure Communications in Wireless-Powered NOMA systems. IEEE Trans. Veh. Technol. 70(2), 1978–1983 (2021)

    Article  Google Scholar 

  32. Qin, X., Liu, Z., Liu, Y., Liu, S., Yang, B., Yin, L., Liu, M., Zheng, W.: User OCEAN personality model construction method using a BP neural network. Electronics. 11(19), 3022 (2022)

    Article  Google Scholar 

  33. Guo, L., Ye, C., Ding, Y., Wang, P.: Allocation of centrally switched fault current limiters enabled by 5G in transmission system. IEEE Trans. Power Deliv. 36(5), 3231–3241 (2021)

    Article  Google Scholar 

  34. Kong, H., Lu, L., Yu, J., Chen, Y., Tang, F.: Continuous authentication through finger gesture interaction for smart homes using WiFi. IEEE Trans. Mob. Comput. 20(11), 3148–3162 (2021)

    Article  Google Scholar 

  35. Chen, Z., Tang, J., Zhang, X.Y., So, D.K.C., Jin, S., et al.: Hybrid evolutionary-based Sparse Channel estimation for IRS-assisted mmWave MIMO systems. IEEE Trans. Wirel. Commun. 21(3), 1586–1601 (2022)

    Article  Google Scholar 

  36. Dai, X., Xiao, Z., Jiang, H., Alazab, M., Lui, J.C.S., Dustdar, S., et al.: Task co-offloading for D2D-assisted Mobile edge computing in industrial internet of things. IEEE Trans. Indust. Inf. 19(1), 480–490 (2023)

    Article  Google Scholar 

  37. Fu, Y., Li, C., Yu, F.R., Luan, T.H., Zhao, P., et al.: A survey of blockchain and intelligent networking for the metaverse. IEEE Internet Things J. 10(4) (2023)

  38. Cao, B., Sun, Z., Zhang, J., Gu, Y.: Resource allocation in 5G IoV architecture based on SDN and fog-cloud computing. IEEE Trans. Intell. Transp. Syst. 22(6), 3832–3840 (2021)

    Article  Google Scholar 

  39. Liu, M., Gu, Q., Yang, B., Yin, Z., Liu, S., Yin, L., ... Zheng, W.: Kinematics model optimization algorithm for six degrees of freedom parallel platform. Appl. Sci., 13(5), (2023)

  40. Zhang, J., Liu, Y., Li, Z., Lu, Y.: “Forecast-assisted service function chain dynamic deployment for SDN/NFV-enabled cloud management systems,” IEEE Syst. J., (2023)

  41. Xu, Y., Cheng, P., Chen, Z., Ding, M., Li, Y., Vucetic, B.: Task offloading for large-scale asynchronous Mobile edge computing: an index policy approach. IEEE Trans. Signal Process. 69, 401–416 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  42. Chen, X., Zhang, H., Wu, C., Mao, S., Ji, Y., Bennis, M.: Optimized computation offloading performance in virtual edge computing systems via deep reinforcement learning. IEEE Internet Things J. 6, 4005–4018 (2018)

    Article  Google Scholar 

  43. Zhu, T., Shi, T., Li, J., Cai, Z., Zhou, X.: Task scheduling in deadline-aware Mobile edge computing systems. IEEE Internet Things J. 6, 4854–4866 (2018)

    Article  Google Scholar 

  44. Xu, X., Li, H., Xu, W., Liu, Z., Yao, L., Dai, F.: Artificial intelligence for edge service optimization in internet of vehicles: a survey. Tsinghua Sci. Technol. 22, 270–287 (2022)

    Article  Google Scholar 

  45. Johari, N. M., Nohuddin, P. N., Baharin, A. H. A., Yakob, N. A., Ebadi, M. J.: Features Requirement Elicitation Process for Designing a Chatbot Application. IET Networks, (2022)

  46. Chiniforooshan Esfahani, I.: A data-driven physics-informed neural network for predicting the viscosity of nanofluids. AIP. Adv. 13(2), 02520 (2023)

    Article  Google Scholar 

  47. Alimohammadirokni, M., Emadlou, A., Yuan, J.J.: The strategic resources of a gastronomy creative city: the case of San Antonio, Texas. J. Gastronomy Tour. 5(4), 237–252 (2021)

    Article  Google Scholar 

  48. Hasselt, H.: Double Q-learning. Adv. Neural Inf. Proces. Syst., 23, (2010)

  49. Bozorgkhou, H., Alimohammadirokni, M.: Studying and investigating the impact of marketing mix factors on e-purchase via smart phones (case study: Digikala corporation). Nexo Revista Científica. 35(04), 992–1003 (2022)

    Article  Google Scholar 

  50. Tang, M.; Wong, V.: Deep Reinforcement Learning for Task Offloading in Mobile Edge Computing Systems. IEEE Transactions on Mobile Computing. 21(6), 1985–1997 (2020)

Download references

Funding

This research received no specific grant from any funding agency.

Author information

Authors and Affiliations

  1. School of Information Engineering, Shanxi Vocational University of Engineering Science and Technology, TauYuan, 030000, China

    YanRu Dong

  2. Department of Computer Engineering, College of Computers and Information Technology, University of Tabuk, 47713, Tabuk, Saudi Arabia

    Ahmed M. Alwakeel, Mohammed M. Alwakeel & Lubna A. Alharbi

  3. Department of Computer Science, College of Computer and Information Sciences, Majmaah University, Al-Majmaah, 11952, Saudi Arabia

    Sara A Althubiti

Authors
  1. YanRu Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ahmed M. Alwakeel
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mohammed M. Alwakeel
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lubna A. Alharbi
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Sara A Althubiti
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

YanRu Dong: Conceptualization, Methodology, Formal analysis, Supervision, Writing - original draft, Writing - review & editing.

Ahmed M. Alwakeel: Writing - original draft, Writing - review & editing.

Mohammed M. Alwakeel: Investigation, Data Curation, Validation, Resources, Writing - review & editing.

Lubna A. Alharbi: Project administration, Investigation, Writing - review & editing.

Sara A Althubiti: Software, Visualization, Writing - original draft.

Corresponding author

Correspondence to YanRu Dong.

Ethics declarations

Ethics Approval and Consent to Participate

Not applicable.

Consent for Publication

Not applicable.

Competing Interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dong, Y., Alwakeel, A.M., Alwakeel, M.M. et al. A Heuristic Deep Q Learning for Offloading in Edge Devices in 5 g Networks. J Grid Computing 21, 37 (2023). https://doi.org/10.1007/s10723-023-09667-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10723-023-09667-w

Keywords

Search

Navigation