Advertisement

A Reinforcement Learning Based Task Offloading Scheme for Vehicular Edge Computing Network

  • Jie Zhang
  • Hongzhi Guo
  • Jiajia LiuEmail author
Conference paper
Part of the Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering book series (LNICST, volume 287)

Abstract

Recently, the trends of automation and intelligence in vehicular networks have led to the emergence of intelligent connected vehicles (ICVs), and various intelligent applications like autonomous driving have also rapidly developed. Usually, these applications are compute-intensive, and require large amounts of computation resources, which conflicts with resource-limited vehicles. This contradiction becomes a bottleneck in the development of vehicular networks. To address this challenge, the researchers combined mobile edge computing (MEC) with vehicular networks, and proposed vehicular edge computing networks (VECNs). The deploying of MEC servers near the vehicles allows compute-intensive applications to be offloaded to MEC servers for execution, so as to alleviate vehicles’ computational pressure. However, the high dynamic feature which makes traditional optimization algorithms like convex/non-convex optimization less suitable for vehicular networks, often lacks adequate consideration in the existing task offloading schemes. Toward this end, we propose a reinforcement learning based task offloading scheme, i.e., a deep Q learning algorithm, to solve the delay minimization problem in VECNs. Extensive numerical results corroborate the superior performance of our proposed scheme on reducing the processing delay of vehicles’ computation tasks.

Keywords

Vehicular edge computing networks Mobile edge computing Reinforcement learning 

References

  1. 1.
    Zhu, H., Yuen, K., Mihaylova, L., Leung, H.: Overview of environment perception for intelligent vehicles. IEEE Trans. Intell. Transp. Syst. 18(10), 2584–2601 (2017).  https://doi.org/10.1109/TITS.2017.2658662CrossRefGoogle Scholar
  2. 2.
    Vegni, A.M., Loscrí, V.: A survey on vehicular social networks. IEEE Commun. Surv. Tutor. 17(4), 2397–2419 (2015).  https://doi.org/10.1109/COMST.2015.2453481CrossRefGoogle Scholar
  3. 3.
    Shojafar, M., Cordeschi, N., Baccarelli, E.: Energy-efficient adaptive resource management for real-time vehicular cloud services. IEEE Trans. Cloud Comput. 7(1), 196–209 (2019).  https://doi.org/10.1109/TCC.2016.2551747CrossRefGoogle Scholar
  4. 4.
    Guo, H., Liu, J., Zhang, J., Sun, W., Kato, N.: Mobile-edge computation offloading for ultradense iot networks. IEEE Internet Things J. 5(6), 4977–4988 (2018).  https://doi.org/10.1109/JIOT.2018.2838584CrossRefGoogle Scholar
  5. 5.
    Wang, S., Zhang, X., Zhang, Y., Wang, L., Yang, J., Wang, W.: A survey on mobile edge networks: convergence of computing, caching and communications. IEEE Access 5, 6757–6779 (2017).  https://doi.org/10.1109/ACCESS.2017.2685434CrossRefGoogle Scholar
  6. 6.
    Zhang, K., Mao, Y., Leng, S., Vinel, A., Zhang, Y.: Delay constrained offloading for mobile edge computing in cloud-enabled vehicular networks. In: 2016 8th International Workshop on Resilient Networks Design and Modeling (RNDM), pp. 288–294, September 2016.  https://doi.org/10.1109/RNDM.2016.7608300
  7. 7.
    Liu, Y., Wang, S., Huang, J., Yang, F.: A computation offloading algorithm based on game theory for vehicular edge networks. In: 2018 IEEE International Conference on Communications (ICC), pp. 1–6, May 2018.  https://doi.org/10.1109/ICC.2018.8422240
  8. 8.
    Dai, Y., Xu, D., Maharjan, S., Zhang, Y.: Joint load balancing and offloading in vehicular edge computing and networks. IEEE Internet Things J. 1 (2019).  https://doi.org/10.1109/JIOT.2018.2876298CrossRefGoogle Scholar
  9. 9.
    Zhang, K., Mao, Y., Leng, S., He, Y., Zhang, Y.: Mobile-edge computing for vehicular networks: a promising network paradigm with predictive off-loading. IEEE Veh. Technol. Mag. 12(2), 36–44 (2017).  https://doi.org/10.1109/MVT.2017.2668838CrossRefGoogle Scholar
  10. 10.
    Guo, H., Zhang, J., Liu, J.: Fiwi-enhanced vehicular edge computing networks: collaborative task offloading. IEEE Veh. Technol. Mag. 14(1), 45–53 (2019).  https://doi.org/10.1109/MVT.2018.2879537CrossRefGoogle Scholar
  11. 11.
    Zheng, Q., Zheng, K., Zhang, H., Leung, V.C.M.: Delay-optimal virtualized radio resource scheduling in software-defined vehicular networks via stochastic learning. IEEE Trans. Veh. Technol. 65(10), 7857–7867 (2016).  https://doi.org/10.1109/TVT.2016.2538461CrossRefGoogle Scholar
  12. 12.
    He, Y., Zhao, N., Yin, H.: Integrated networking, caching, and computing for connected vehicles: a deep reinforcement learning approach. IEEE Trans. Veh. Technol. 67(1), 44–55 (2018).  https://doi.org/10.1109/TVT.2017.2760281CrossRefGoogle Scholar
  13. 13.
    Ye, H., Li, G.Y.: Deep reinforcement learning for resource allocation in V2V communications. In: 2018 IEEE International Conference on Communications (ICC), pp. 1–6, May 2018.  https://doi.org/10.1109/ICC.2018.8422586
  14. 14.
    Guo, H., Liu, J.: Collaborative computation offloading for multiaccess edge computing over fiber-wireless networks. IEEE Trans. Veh. Technol. 67(5), 4514–4526 (2018).  https://doi.org/10.1109/TVT.2018.2790421CrossRefGoogle Scholar
  15. 15.
    Li, J., Gao, H., Lv, T., Lu, Y.: Deep reinforcement learning based computation offloading and resource allocation for mec. In: 2018 IEEE Wireless Communications and Networking Conference (WCNC), pp. 1–6, April 2018.  https://doi.org/10.1109/WCNC.2018.8377343

Copyright information

© ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering 2019

Authors and Affiliations

  1. 1.School of Cyber EngineeringXidian UniversityXi’anChina
  2. 2.School of CybersecurityNorthwestern Polytechnical UniversityXi’anChina

Personalised recommendations