A Reinforcement Learning Based Task Offloading Scheme for Vehicular Edge Computing Network

  • Jie Zhang
  • Hongzhi Guo
  • Jiajia LiuEmail author
Conference paper
Part of the Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering book series (LNICST, volume 287)


Recently, the trends of automation and intelligence in vehicular networks have led to the emergence of intelligent connected vehicles (ICVs), and various intelligent applications like autonomous driving have also rapidly developed. Usually, these applications are compute-intensive, and require large amounts of computation resources, which conflicts with resource-limited vehicles. This contradiction becomes a bottleneck in the development of vehicular networks. To address this challenge, the researchers combined mobile edge computing (MEC) with vehicular networks, and proposed vehicular edge computing networks (VECNs). The deploying of MEC servers near the vehicles allows compute-intensive applications to be offloaded to MEC servers for execution, so as to alleviate vehicles’ computational pressure. However, the high dynamic feature which makes traditional optimization algorithms like convex/non-convex optimization less suitable for vehicular networks, often lacks adequate consideration in the existing task offloading schemes. Toward this end, we propose a reinforcement learning based task offloading scheme, i.e., a deep Q learning algorithm, to solve the delay minimization problem in VECNs. Extensive numerical results corroborate the superior performance of our proposed scheme on reducing the processing delay of vehicles’ computation tasks.


Vehicular edge computing networks Mobile edge computing Reinforcement learning 


  1. 1.
    Zhu, H., Yuen, K., Mihaylova, L., Leung, H.: Overview of environment perception for intelligent vehicles. IEEE Trans. Intell. Transp. Syst. 18(10), 2584–2601 (2017). Scholar
  2. 2.
    Vegni, A.M., Loscrí, V.: A survey on vehicular social networks. IEEE Commun. Surv. Tutor. 17(4), 2397–2419 (2015). Scholar
  3. 3.
    Shojafar, M., Cordeschi, N., Baccarelli, E.: Energy-efficient adaptive resource management for real-time vehicular cloud services. IEEE Trans. Cloud Comput. 7(1), 196–209 (2019). Scholar
  4. 4.
    Guo, H., Liu, J., Zhang, J., Sun, W., Kato, N.: Mobile-edge computation offloading for ultradense iot networks. IEEE Internet Things J. 5(6), 4977–4988 (2018). Scholar
  5. 5.
    Wang, S., Zhang, X., Zhang, Y., Wang, L., Yang, J., Wang, W.: A survey on mobile edge networks: convergence of computing, caching and communications. IEEE Access 5, 6757–6779 (2017). Scholar
  6. 6.
    Zhang, K., Mao, Y., Leng, S., Vinel, A., Zhang, Y.: Delay constrained offloading for mobile edge computing in cloud-enabled vehicular networks. In: 2016 8th International Workshop on Resilient Networks Design and Modeling (RNDM), pp. 288–294, September 2016.
  7. 7.
    Liu, Y., Wang, S., Huang, J., Yang, F.: A computation offloading algorithm based on game theory for vehicular edge networks. In: 2018 IEEE International Conference on Communications (ICC), pp. 1–6, May 2018.
  8. 8.
    Dai, Y., Xu, D., Maharjan, S., Zhang, Y.: Joint load balancing and offloading in vehicular edge computing and networks. IEEE Internet Things J. 1 (2019). Scholar
  9. 9.
    Zhang, K., Mao, Y., Leng, S., He, Y., Zhang, Y.: Mobile-edge computing for vehicular networks: a promising network paradigm with predictive off-loading. IEEE Veh. Technol. Mag. 12(2), 36–44 (2017). Scholar
  10. 10.
    Guo, H., Zhang, J., Liu, J.: Fiwi-enhanced vehicular edge computing networks: collaborative task offloading. IEEE Veh. Technol. Mag. 14(1), 45–53 (2019). Scholar
  11. 11.
    Zheng, Q., Zheng, K., Zhang, H., Leung, V.C.M.: Delay-optimal virtualized radio resource scheduling in software-defined vehicular networks via stochastic learning. IEEE Trans. Veh. Technol. 65(10), 7857–7867 (2016). Scholar
  12. 12.
    He, Y., Zhao, N., Yin, H.: Integrated networking, caching, and computing for connected vehicles: a deep reinforcement learning approach. IEEE Trans. Veh. Technol. 67(1), 44–55 (2018). Scholar
  13. 13.
    Ye, H., Li, G.Y.: Deep reinforcement learning for resource allocation in V2V communications. In: 2018 IEEE International Conference on Communications (ICC), pp. 1–6, May 2018.
  14. 14.
    Guo, H., Liu, J.: Collaborative computation offloading for multiaccess edge computing over fiber-wireless networks. IEEE Trans. Veh. Technol. 67(5), 4514–4526 (2018). Scholar
  15. 15.
    Li, J., Gao, H., Lv, T., Lu, Y.: Deep reinforcement learning based computation offloading and resource allocation for mec. In: 2018 IEEE Wireless Communications and Networking Conference (WCNC), pp. 1–6, April 2018.

Copyright information

© ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering 2019

Authors and Affiliations

  1. 1.School of Cyber EngineeringXidian UniversityXi’anChina
  2. 2.School of CybersecurityNorthwestern Polytechnical UniversityXi’anChina

Personalised recommendations