Skip to main content

Advertisement

SpringerLink
Log in

Task offloading method of edge computing in internet of vehicles based on deep reinforcement learning

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

Compared with the traditional network tasks, the emerging Internet of Vehicles (IoV) technology has higher requirements for network bandwidth and delay. However, due to the limitation of computing resources and battery capacity of existing mobile devices, it is hard to meet the above requirements. How to complete task offloading and calculation with lower task delay and lower energy consumption is the most important issue. Aiming at the task offloading system of the IoV, this paper considers the situation of multiple MEC servers when modeling, and proposes a dynamic task offloading scheme based on deep reinforcement learning. It improves the traditional Q-Learning algorithm and combines deep learning with reinforcement learning to avoid dimensional disaster in the Q-Learning algorithm. Simulation results show that the proposed algorithm has better performance on delay, energy consumption, and total system overhead under the different number of tasks and wireless channel bandwidth.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

References

  1. Tan, H.Z.: Overall computing offloading strategy based on deep reinforcement learning in vehicle fog computing. J. Eng. 2020(11), 1080 (2020). https://doi.org/10.1049/joe.2020.0134

    Article  Google Scholar 

  2. Ge, H., Zhang, T.: New multi-hop clustering algorithm for vehicular Ad Hoc networks. IEEE Trans. Intell. Transp. Syst. 20(4), 1517–1530 (2019)

    Article  Google Scholar 

  3. Gao, J.X.: Novel approach of distributed & adaptive trust metrics for MANET. Wirel. Netw. 25(6), 3587–3603 (2019)

    Article  Google Scholar 

  4. Samie, F., Tsoutsouras, V., Bauer, L. et al.: Computation offloading and resource allocation for low-power IoT edge devices. In: IEEE World Forum on Internet of Things (WF-IoT), vol. 1, no. 1, pp. 7–12. IEEE (2016)

  5. Mao, Y., Zhang, J., Letaief, K.B.: Dynamic computation offloading for mobile-edge compu-ting with energy harvesting devices. IEEE J. Sel. Areas Commun. 34(12), 3590–3605 (2016)

    Article  Google Scholar 

  6. Liu, Y., Wang, S., Yang, F.: Poster abstract: a multi-user computation offloading algorithm based on game theory in mobile cloud compu-ting. Edge Comput. 1(1), 93–94 (2016)

    Google Scholar 

  7. Chen, M., Hao, Y.: Task offloading for mobile edge computing in software defined ultra-dense network. IEEE J. Sel. Areas Commun. 36(3), 587–597 (2018)

    Article  Google Scholar 

  8. Mukherjee, M., Shu, L., Wang, D.: Survey of fog computing: fundamental, network applications, and research challenges. IEEE Commun. Surv. Tutor. 20(3), 1826–1857 (2018)

    Article  Google Scholar 

  9. Zhang, T., Liu, X.: Novel self- adaptive routing service algorithm for application in VANET. Appl. Intell. 49, 1866–1879 (2018)

    Article  Google Scholar 

  10. Zhang, D.G., Li, G., Zheng, K.: An energy- balanced routing method based on forward-aware factorfor Wireless Sensor Network. IEEE Trans. Ind. Inform. 10(1), 766–773 (2014)

    Article  Google Scholar 

  11. Cui, Y.Y.: New quantum-genetic based OLSR protocol (QG-OLSR) for mobile ad hoc network. Appl. Soft Comput. 80(7), 285–296 (2019). https://doi.org/10.1016/j.asoc.2019.03.053

    Article  Google Scholar 

  12. Wang, X., Song, X.D.: A novel approach to mapped correlation of ID for RFID anti-collision. IEEE Trans. Serv. Comput. 7(4), 741–748 (2014)

    Article  MathSciNet  Google Scholar 

  13. Zheng, K., Zhang, T.: A novel multicast R-outing method with minimum transmission forWSN of cloud computing service. Soft Comput. 19(7), 1817–1827 (2015)

    Article  Google Scholar 

  14. Gong, C.L.: A new algorithm of clusteringAODV based on edge computing strategy in IOV. Wirel. Netw. 27(4), 2891–2908 (2021). https://doi.org/10.1007/s11276-021-02624-z

    Article  Google Scholar 

  15. Zhang, D.G.: A new approach and system for attentive mobile learning based on seamless migration. Appl. Intell. 36(1), 75–89 (2012)

    Article  Google Scholar 

  16. Zheng, K., Zheng, K., Zhao, D.X., et al.: Novel Quick Start (QS) Method for Optimization of TCP. Wirel. Netw. 22(1), 211–222 (2016)

    Article  Google Scholar 

  17. Piao, M.J., Zhang, T.: New Algorithm of multi-strategy channel allocation for edge computing. AEUE-Int. J. Electron. Commun. 126(11), 1–15 (2020). https://doi.org/10.1016/j.aeue.2020.153372

    Article  Google Scholar 

  18. Zhang, T., Zhang, J.: A kind of effective data aggregating method based on compressive sensing for wireless sensor network. EURASIP J. Wirel. Commun. Netw. 2018(159), 1–15 (2018). https://doi.org/10.1186/s13638-018-1176-4

    Article  Google Scholar 

  19. Zhang, T., Dong, Y.: Novel optimized link state routing protocol based on quantum genetic strategy for mobile learning. J. Netw. Comput. Appl. 2018(122), 37–49 (2018). https://doi.org/10.1016/j.jnca.2018.07.018

    Article  Google Scholar 

  20. Chen, X.: Decentralized computation offloading game for mobile cloud. IEEE Trans. Parallel Distrib. Syst. Comput. 26(4), 974–983 (2015)

    Article  Google Scholar 

  21. Liu, S., Zhang, T.: Novel unequal clustering routing protocol considering energy balancing based on network partition & distance for mobile education. J. Netw. Comput. Appl. 88(15), 1–9 (2017)

    Google Scholar 

  22. Zhou, S.: A low duty cycle efficient MAC protocol based on self-adaption and predictive strategy. Mob. Netw. Appl. 23(4), 828–839 (2018)

    Article  Google Scholar 

  23. Niu, H.L., Liu, S.: Novel PEECR-based clustering routing approach. Soft. Comput. 21(24), 7313–7323 (2017). https://doi.org/10.1007/s00500-016-2270-3

    Article  Google Scholar 

  24. Chen, C.: New method of energy efficient subcarrier allocation based on Evolutionary Game Theory. Mob. Netw. Appl. 26(2), 523–536 (2021). https://doi.org/10.1007/s11036-018-1123-y

    Article  Google Scholar 

  25. Liu, S., Liu, X.H.: Novel dynamic s-ource routing protocol (DSR) based on geneticalgorithm-bacterial foraging optimization (GABFO). Int. J. Commun. Syst. 31(18), 1–20 (2018). https://doi.org/10.1002/dac.3824

    Article  Google Scholar 

  26. Tang, Y.M., Cui, Y.Y.: Novel reliable rou-ting method for engineering of internet of veh-icles based on graph theory. Eng. Comput. 36(1), 226–247 (2019). https://doi.org/10.1108/EC-072018-0299

    Article  Google Scholar 

  27. Liu, S., Liu, X.H., Zhang, T.: Dynamic analysis for the average shortest path length ofmobile ad hoc networks under random failure scenarios. IEEE Access 7, 21343–21358 (2019). https://doi.org/10.1109/ACCESS.2019.2896699

    Article  Google Scholar 

  28. Wu, H.: New approach of multi-path reliable transmission for marginal wireless sensor network. Wirel. Netw. 12, 1–15 (2019). https://doi.org/10.1007/s11276-019-02216-y

    Article  Google Scholar 

  29. Liu, X.H.: Novel Best path selection approach based on hybrid improved A* Algorithmand reinforcement learning. Appl. Intell. 51(9), 1–15 (2021)

    Google Scholar 

  30. Liu, S., Liu, X.H., Zhang, T., et al.: Adaptive repair algorithm for TORA routing protocol based on flood control strategy. Comput. Commun. 151(1), 437–448 (2020). https://doi.org/10.1016/j.comcom.2020.01.024

    Article  Google Scholar 

  31. Cui, Y.Y., Zhang, T., Chen, L., et al.: Novel method of mobile edge computation offloading based on evolutionary game strategy for IoT devices. AEU-Int. J. Electron. Commun. 118(5), 1–13 (2020). https://doi.org/10.1016/j.aeue.2020.153134

    Article  Google Scholar 

  32. Chen, L., Zhang, J., Chen, J., et al.: A multi-path routing protocol based on link lifetime and energy consumption prediction for mobile edge compu-ting. IEEE Access 8(1), 69058–69071 (2020). https://doi.org/10.1109/ACCESS.2020.2986078

    Article  Google Scholar 

  33. Wang, Y., Sheng, M., Wang, X., et al.: Mobi-le-edge computing: partial computation offlo-ading using dynamic voltage scaling. IEEE Trans. Commun. 64(10), 4268–4282 (2016)

    Google Scholar 

  34. Wang, C., Liang, C., Yu, F.R., et al.: Computation offloading and resource allocation in wireless cellular networks with mobile edge computing. IEEE Trans. Veh. Technol. 16(8), 4924–4938 (2017)

    Google Scholar 

  35. Wu, Y., Ni, K., Zhang, C., et al.: NOMA assisted multiaccess mobile edge computing a joint optimization of computation offloading and time allocation. IEEE Trans. Veh. Technol. 67(12), 12244–12258 (2018)

    Article  Google Scholar 

  36. Liu, R.Z., Sheng, M., Wu, W.H.: Energy-efficient resource allocation for heterogeneous wireless network with multi-homed user equipment. IEEE Access 6, 14591–14601 (2018). https://doi.org/10.1109/ACCESS.2018.2810216

    Article  Google Scholar 

  37. Liu, D., Wang, L., Chen, Y., et al.: Distributed energy efficient fair user association in massive MIMO enabled HetNets. IEEE Commun. Lett. 19(10), 1770–1773 (2015)

    Article  Google Scholar 

  38. Lyu, X.C., Tian, H., Sengul, C., et al.: Multi-user joint task offloading and resource optimization in proximate clouds. IEEE Trans. Veh. Technol. 66(4), 3435–3447 (2017)

    Article  Google Scholar 

  39. Mahmoodi, S.E., Uma, R.N., Subbalakshmi, K.P.: Optimal joint scheduling and cloud offloading for mobile applications. IEEE Trans. Cloud Comput. 7(2), 301–313 (2016)

    Article  Google Scholar 

  40. Silver, D., Hubert, T., Schrittwieser, J., et al.: A general rein-forcement learning algorithm that masters chess, shogi, and go through self-play. Science 362(6419), 1140–1144 (2018)

    Article  MathSciNet  Google Scholar 

  41. Abbas, N., Zhang, Y., Taherkordi, A., et al.: Mobile edge computing: a survey. IEEE Internet Things J. 5(1), 450–465 (2018)

    Article  Google Scholar 

  42. Zhang, T.: A new method of data missing estimation with FNN-based tensor heterogeneous ensemble learning for internet of vehicle. Neurocomputing 420(1), 98–110 (2021). https://doi.org/10.1016/j.neucom.2020.09.042

    Article  Google Scholar 

  43. Zhang, T.: A kind of novel method of power allocation with limited cross-tier interference for CRN. IEEE Access 7(1), 82571–82583 (2019). https://doi.org/10.1109/ACCESS.2019.2921310

    Article  Google Scholar 

  44. Liu, X.H.: A new algorithm of the best path selection based on machine learning. IEEE Access 7(1), 126913–126928 (2019). https://doi.org/10.1109/ACCESS.2019.2939423

    Article  Google Scholar 

  45. Zhao, P.Z.: A new method of mobile ad hoc network routing based on greed forwarding improvement strategy. IEEE Access 7(1), 158514–158524 (2019). https://doi.org/10.1109/ACCESS.2019.2950266

    Article  Google Scholar 

  46. Venkatraman, B., Moayad, A., Martin, R.: Intelligent resource management at the edge for ubiquitous IoT: an SDN-based federated learning approach. IEEE Netw. 35(5), 114–121 (2021)

    Article  Google Scholar 

  47. Venkatraman, B., Moayad, A., Martin, R.: FedCo: a federated learning controller for content management in multi-party edge systems. In: ICCCN2021, vol. 1, no. 1, pp. 1–9 (2021)

Download references

Funding

This research work is supported by Graduate Students' Innovative Plan Program (No.2020YJSB079), National Natural Science Foundation of China (Grant No. 61571328), Tianjin Key Natural Science Foundation (No. 13JCZDJC34600, 18JCZDJC96800, No.18JCYBJC19300), Major projects of science and technology in Tianjin (No.15ZXDSGX00050), Training plan of Tianjin University Innovation Team (No.TD12-5016, No.TD13- 5025), Major projects of science and technology for their services in Tianjin (No.16ZXFWGX00010, No.17YFZ CGX00360), the Key Subject Foundation of Tianjin (15JCYBJC46500), Training plan of Tianjin 131 Innovation Talent Team (No. TD2015-23).

Author information

Authors and Affiliations

  1. Tianjin Key Lab of Intelligent Computing and Novel Software Technology, Tianjin University of Technology, Tianjin, 300384, China

    Degan Zhang, Lixiang Cao, Haoli Zhu, Jinyu Du & Kaiwen Jiang

  2. School of Sports Economics and Management, Tianjin University of Sport, Tianjin, 301617, China

    Ting Zhang

Authors
  1. Degan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lixiang Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Haoli Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ting Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jinyu Du
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Kaiwen Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

DZ: Conceptualization, Writing-Reviewing and Editing, Methodology. LC: Revision, Supervision, Editing. HZ: Data curation, Writing-Original draft preparation, Editing. TZ: Visualization, Investigation. JD: Supervision, Editing. KJ: Software, Validation, Editing.

Corresponding author

Correspondence to Kaiwen Jiang.

Ethics declarations

Conflict of interest

Author Degan Zhang, Lixiang Cao, Haoli Zhu, Ting Zhang, Jinyu Du, and Kaiwen Jiang declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, D., Cao, L., Zhu, H. et al. Task offloading method of edge computing in internet of vehicles based on deep reinforcement learning. Cluster Comput 25, 1175–1187 (2022). https://doi.org/10.1007/s10586-021-03532-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10586-021-03532-9

Keywords

Search

Navigation