Skip to main content
SpringerLink
Log in

DeepMECagent: multi-agent computing resource allocation for UAV-assisted mobile edge computing in distributed IoT system

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

The proliferation of Internet-of-Things (IoTs) devices provides a promising platform for various intelligent applications such as virtual reality. However, because of the limited onboard computation resource of IoT devices, the computation task suffers from long latency. Mobile edge computing (MEC), which is a critical technology, allows offloading the computation tasks to an edge server to alleviate the shortage of computation resources and accelerate the computation tasks of IoT devices. Owing to the flexibility and mobility advantages of unmanned aerial vehicles (UAVs), UAV-assisted MEC has been widely researched. However, existing studies mostly explore a centralized offloading strategy. Therefore, when the number of IoT devices increases, the performance of the centralized strategy is reduced. The present study explores an intelligent strategy to minimize computation latency using a distributed algorithm. We develop a distributed algorithm named DeepMECagent based on deep reinforcement learning to optimize the computation offloading with minimum computation latency. In the considered scenario, a UAV functions as an aerial edge server to collect and process the computation tasks offloaded by ground IoT devices.The simulation results demonstrate the effective-ness of the proposed approach for minimizing the computation latency, where the computation latency of the proposed algorithm improves 39.71%,87.42%, and 88.55%, respectively, while compared with the centralized-DQN, Q-table, and the random algorithm. Given the expiration time as 1 second, the number of completed tasks within the expiration time of the proposed DeepMECagent is around 2 × and 1.25 × compared with the random algorithm and the Q-table algorithm, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. The future of iot miniguide: The burgeoning iot market continues. https://www.cisco.com/c/en/us/solutions/internet-of-things/future-of-iot.html

  2. Abbas N, Zhang Y, Taherkordi A, Skeie T (2017) Mobile edge computing: a survey. IEEE Internet of Things Journal 5(1):450–465

    Article  Google Scholar 

  3. Arulkumaran K, Deisenroth MP, Brundage M, Bharath AA (2017) Deep reinforcement learning: a brief survey. IEEE Signal Proc Mag 34(6):26–38

    Article  Google Scholar 

  4. Dan L, Xin C, Chongwei H, Liangliang J (2015) Intelligent agriculture greenhouse environment monitoring system based on iot technology. In: 2015 International conference on intelligent transportation, big data and smart city. IEEE, pp 487–490

  5. Diao X, Zheng J, Cai Y, Wu Y, Anpalagan A (2019) Fair data allocation and trajectory optimization for uav-assisted mobile edge computing. IEEE Commun Lett 23(12):2357–2361

    Article  Google Scholar 

  6. Erdelj M, Król M, Natalizio E (2017) Wireless sensor networks and multi-uav systems for natural disaster management. Comput Netw 124:72–86

    Article  Google Scholar 

  7. Guo H, Liu J (2019) Uav-enhanced intelligent offloading for internet of things at the edge. IEEE Transactions on Industrial Informatics 16(4):2737–2746

    Article  Google Scholar 

  8. Hu W, Zhou Y, Zhang Z, Fujita H (2021) Model predictive control for hybrid levitation systems of maglev trains with state constraints. IEEE Trans Veh Technol 70(10):9972–9985

    Article  Google Scholar 

  9. Hua M, Wang Y, Li C, Huang Y, Yang L (2019) Uav-aided mobile edge computing systems with one by one access scheme. IEEE Transactions on Green Communications and Networking 3(3):664–678

    Article  Google Scholar 

  10. Ji J, Zhu K, Yi C, Niyato D (2020) Energy consumption minimization in uav-assisted mobile-edge computing systems: Joint resource allocation and trajectory design. IEEE Internet of Things Journal 8(10):8570–8584

    Article  Google Scholar 

  11. Li B, Fei Z, Zhang Y (2018) Uav communications for 5g and beyond: Recent advances and future trends. IEEE Internet of Things Journal 6(2):2241–2263

    Article  Google Scholar 

  12. Li M, Cheng N, Gao J, Wang Y, Zhao L, Shen X (2020) Energy-efficient uav-assisted mobile edge computing: Resource allocation and trajectory optimization. IEEE Trans Veh Technol 69(3):3424–3438

    Article  Google Scholar 

  13. Li Y (2017) Deep reinforcement learning: An overview. arXiv:1701.07274

  14. Liu Y, Qiu M, Hu J, Yu H (2020) Incentive uav-enabled mobile edge computing based on microwave power transmission. IEEE Access 8:28,584–28,593

    Article  Google Scholar 

  15. Lyu J, Zeng Y, Zhang R (2016) Cyclical multiple access in uav-aided communications: a throughput-delay tradeoff. IEEE Wireless Communications Letters 5(6):600–603

    Article  Google Scholar 

  16. Mach P, Becvar Z (2017) Mobile edge computing: a survey on architecture and computation offloading. IEEE Communications Surveys & Tutorials 19(3):1628–1656

    Article  Google Scholar 

  17. Mao Y, You C, Zhang J, Huang K, Letaief KB (2017) A survey on mobile edge computing: The communication perspective. IEEE Communications Surveys & Tutorials 19(4):2322–2358

    Article  Google Scholar 

  18. Miraz MH, Ali M, Excell PS, Picking R (2018) Internet of nano-things, things and everything: future growth trends. Future Internet 10(8):68

    Article  Google Scholar 

  19. Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, Graves A, Riedmiller M, Fidjeland AK, Ostrovski G et al (2015) Human-level control through deep reinforcement learning. Nature 518(7540):529–533

    Article  Google Scholar 

  20. Mohammed A, Nahom H, Tewodros A, Habtamu Y, Hayelom G (2020) Deep reinforcement learning for computation offloading and resource allocation in blockchain-based multi-uav-enabled mobile edge computing. In: 2020 17Th international computer conference on wavelet active media technology and information processing (ICCWAMTIP). IEEE, pp 295–299

  21. Morales M (2020) Grokking deep reinforcement learning manning publications

  22. Mozaffari M, Kasgari ATZ, Saad W, Bennis M, Debbah M (2018) Beyond 5g with uavs: Foundations of a 3d wireless cellular network. IEEE Trans Wirel Commun 18(1):357–372

    Article  Google Scholar 

  23. Mukherjee M, Kumar V, Lat A, Guo M, Matam R, Lv Y (2020) Distributed deep learning-based task offloading for uav-enabled mobile edge computing. In: IEEE INFOCOM 2020-IEEE Conference on computer communications workshops (INFOCOM WKSHPS). IEEE, pp 1208–1212

  24. Nguyen V, Khanh TT, Van Nam P, Thu NT, Hong CS, Huh EN (2020) Towards flying mobile edge computing. In: 2020 International conference on information networking (ICOIN). IEEE, pp 723–725

  25. Poongodi T, Krishnamurthi R, Indrakumari R, Suresh P, Balusamy B (2020) Wearable devices and iot. In: A handbook of internet of things in biomedical and cyber physical system. Springer, pp 245–273

  26. Ren T, Niu J, Dai B, Liu X, Hu Z, Xu M, Guizani M (2021) Enabling efficient scheduling in large-scale uav-assisted mobile edge computing via hierarchical reinforcement learning. IEEE Internet of Things Journal

  27. Ryu JW, Pham QV, Luan HN, Hwang WJ, Kim JD, Lee JT (2019) Multi-access edge computing empowered heterogeneous networks: a novel architecture and potential works. Symmetry 11(7):842

    Article  Google Scholar 

  28. Tokic M, Palm G (2011) Value-difference based exploration: adaptive control between epsilon-greedy and softmax. In: Annual conference on artificial intelligence. Springer, pp 335–346

  29. Wang L, Huang P, Wang K, Zhang G, Zhang L, Aslam N, Yang K (2019) Rl-based user association and resource allocation for multi-uav enabled mec. In: 2019 15Th international wireless communications & mobile computing conference (IWCMC). IEEE, pp 741–746

  30. Wang L, Wang K, Pan C, Xu W, Aslam N, Hanzo L (2020) Multi-agent deep reinforcement learning-based trajectory planning for multi-uav assisted mobile edge computing. IEEE Transactions on Cognitive Communications and Networking 7(1):73–84

    Article  Google Scholar 

  31. Watkins CJ, Dayan P (1992) Q-learning. Machine Learning 8(3-4):279–292

    Article  MATH  Google Scholar 

  32. Yang D, Wu Q, Zeng Y, Zhang R (2018) Energy tradeoff in ground-to-uav communication via trajectory design. IEEE Trans Veh Technol 67(7):6721–6726

    Article  Google Scholar 

  33. Zeng Y, Zhang R (2017) Energy-efficient uav communication with trajectory optimization. IEEE Trans Wirel Commun 16(6):3747–3760

    Article  Google Scholar 

  34. Zhang J, Zhou L, Tang Q, Ngai ECH, Hu X, Zhao H, Wei J (2018) Stochastic computation offloading and trajectory scheduling for uav-assisted mobile edge computing. IEEE Internet of Things Journal 6(2):3688–3699

    Article  Google Scholar 

  35. Zhang K, Gui X, Ren D, Li D (2020) Energy–latency tradeoff for computation offloading in uav-assisted multiaccess edge computing system. IEEE Internet of Things Journal 8(8):6709–6719

    Article  Google Scholar 

  36. Zhang L, Zhang ZY, Min L, Tang C, Zhang HY, Wang YH, Cai P (2021) Task offloading and trajectory control for uav-assisted mobile edge computing using deep reinforcement learning. IEEE Access 9:53,708–53,719

    Article  Google Scholar 

  37. Zhang L, Zhao Z, Wu Q, Zhao H, Xu H, Wu X (2018) Energy-aware dynamic resource allocation in uav assisted mobile edge computing over social internet of vehicles. IEEE Access 6:56,700–56,715

    Article  Google Scholar 

  38. Zhang S, Zeng Y, Zhang R (2018) Cellular-enabled uav communication: Trajectory optimization under connectivity constraint. In: 2018 IEEE International conference on communications (ICC). IEEE, pp 1–6

  39. Zhang T, Xu Y, Loo J, Yang D, Xiao L (2019) Joint computation and communication design for uav-assisted mobile edge computing in iot. IEEE Transactions on Industrial Informatics 16(8):5505–5516

    Article  Google Scholar 

  40. Zhang Y, Zhou Y, Lu H, Fujita H (2021) Cooperative multi-agent actor–critic control of traffic network flow based on edge computing. Futur Gener Comput Syst 123:128–141

    Article  Google Scholar 

  41. Zhang Y, Zhou Y, Lu H, Fujita H (2021) Spark cloud-based parallel computing for traffic network flow predictive control using non-analytical predictive model. IEEE Transactions on Intelligent Transportation Systems

  42. Zhao L, Zhou Y, Lu H, Fujita H (2019) Parallel computing method of deep belief networks and its application to traffic flow prediction. Knowl-Based Syst 163:972–987

    Article  Google Scholar 

  43. Zhou Y, Tao X, Yu Z, Fujita H (2019) Train-movement situation recognition for safety justification using moving-horizon tbm-based multisensor data fusion. Knowl-Based Syst 177:117–126

    Article  Google Scholar 

  44. Zhou Z, Feng J, Tan L, He Y, Gong J (2018) An air-ground integration approach for mobile edge computing in iot. IEEE Commun Mag 56(8):40–47

    Article  Google Scholar 

  45. Zhu S, Gui L, Zhao D, Cheng N, Zhang Q, Lang X (2021) Learning-based computation offloading approaches in uavs-assisted edge computing. IEEE Trans Veh Technol 70(1):928–944

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Qisibo Educational Technology, Beijing, 100084, China

    Xiangxiang Zhang

  2. The University of New South Wales, Sydney, 2052, Australia

    Yichao Wang

Authors
  1. Xiangxiang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yichao Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiangxiang Zhang.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, X., Wang, Y. DeepMECagent: multi-agent computing resource allocation for UAV-assisted mobile edge computing in distributed IoT system. Appl Intell 53, 1180–1191 (2023). https://doi.org/10.1007/s10489-022-03482-8

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-03482-8

Keywords

Search

Navigation