Distributed Deep Learning-based Offloading for Mobile Edge Computing Networks

  • Liang Huang
  • Xu Feng
  • Anqi Feng
  • Yupin Huang
  • Li Ping QianEmail author


This paper studies mobile edge computing (MEC) networks where multiple wireless devices (WDs) choose to offload their computation tasks to an edge server. To conserve energy and maintain quality of service for WDs, the optimization of joint offloading decision and bandwidth allocation is formulated as a mixed integer programming problem. However, the problem is computationally limited by the curse of dimensionality, which cannot be solved by general optimization tools in an effective and efficient way, especially for large-scale WDs. In this paper, we propose a distributed deep learning-based offloading (DDLO) algorithm for MEC networks, where multiple parallel DNNs are used to generate offloading decisions. We adopt a shared replay memory to store newly generated offloading decisions which are further to train and improve all DNNs. Extensive numerical results show that the proposed DDLO algorithm can generate near-optimal offloading decisions in less than one second.


Mobile edge computing Offloading Deep learning Distributed learning 



This work was supported in part by the National Natural Science Foundation of China under Grant No.61502428, and in part by the Zhejiang Provincial Natural Science Foundation of China under Grants No.LR16F010003 and No.LY19F020033.


  1. 1.
    Chiang M, Zhang T (2016) Fog and IoT: an overview of research opportunities. IEEE Internet J 3(6):854–864CrossRefGoogle Scholar
  2. 2.
    Mao Y, You C, Zhang J, Huang K, Letaief K (2017) A survey on mobile edge computing: the communication perspective. IEEE Commun Surv Tutorials 19(4):2322–2358CrossRefGoogle Scholar
  3. 3.
    Abbas N, Zhang Y, Taherkordi A, Skeie T (2018) Mobile edge computing: a survey. IEEE Internet J 5(1):450–465CrossRefGoogle Scholar
  4. 4.
    Chen X, Jiao L, Li W, Fu X (2016) Efficient multi-user computation offloading for mobile-edge cloud computing. IEEE/ACM Trans Netw 24(5):2795–2808CrossRefGoogle Scholar
  5. 5.
    Tran TX, Pompili D (2017) Joint task offloading and resource allocation for multi-server mobile-edge computing networks. arXiv:1705.00704
  6. 6.
    Bi S, Zhang Y (2018) Computation rate maximization for wireless powered mobile-edge computing with binary computation offloading. IEEE Trans Wirel Commun 17(6):4177–4190CrossRefGoogle Scholar
  7. 7.
    Guo S, Xiao B, Yang Y, Yang Y (2016) Energy-efficient dynamic offloading and resource scheduling in mobile cloud computing. In: IEEE international conference on computer communications, pp 1–9Google Scholar
  8. 8.
    Zhang J, Xia W, Zhang Y, Zou Q, Huang B, Yan F, Shen L (2017) Joint offloading and resource allocation optimization for mobile edge computing. In: IEEE global communications conference, pp 1–6Google Scholar
  9. 9.
    Chen M, Liang B, Dong M (2016) Joint offloading decision and bandwidth allocation for multi-user multi-task mobile edge. In: 2016 IEEE international conference on communications, Kuala Lumpur, pp 1–6Google Scholar
  10. 10.
    Qian LP, Zhang Y, Huang H, Wu Y (2013) Demand response management via real-time electricity price control in smart grids. IEEE J Sel Areas Commun 31(7):1268–1280CrossRefGoogle Scholar
  11. 11.
    Qian LP, Wu Y, Zhou H, Shen XS (2017) Joint uplink base station association and power control for small-cell networks with non-orthogonal multiple access. IEEE Trans Wirel Commun 16(9):5567–5582CrossRefGoogle Scholar
  12. 12.
    Qian LP, Wu Y, Zhou H, Shen XS (2017) Dynamic cell association for non-orthogonal multiple-access V2S networks. IEEE J Sel Areas Commun 35(10):2342–2356CrossRefGoogle Scholar
  13. 13.
    LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436CrossRefGoogle Scholar
  14. 14.
    Phaniteja S, Dewangan P, Guhan P, Sarkar A, Krishna K (2017) A deep reinforcement learning approach for dynamically stable inverse kinematics of humanoid robots. In: 2017 IEEE international conference on robotics and biomimetics (ROBIO), Macau, pp 1818–1823Google Scholar
  15. 15.
    Sharma A, Kaushik P (2017) Literature survey of statistical, deep and reinforcement learning in natural language processing. In: International conference on computing, communication and automation, Greater Noida, pp 350–354Google Scholar
  16. 16.
    Mnih V, Kavukcuoglu K, et al. (2015) Human-level control through deep reinforcement learning. Nature 518(7540):529–533CrossRefGoogle Scholar
  17. 17.
    Dulac-Arnold G, Evans R, Hasselt H, Sunehag P, Lillicrap T, Hunt J, Mann T, Weber T, Degris T, Coppin B (2015) Deep reinforcement learning in large discrete action spaces. arXiv:1512.07679
  18. 18.
    Zhang C, Patras P, Haddadi H (2018) Deep learning in mobile and wireless networking: a survey. arXiv:1803.04311
  19. 19.
    Sun H, Chen X, Shi Q, Hong M, Fu X, Sidiropoulos ND (2017) Learning to optimize: Training deep neural networks for wireless resource management. In: Proceedings of IEEE international workshop on signal processing advances in wireless communications, pp 1–6Google Scholar
  20. 20.
    Xu Z, Wang Y, Tang J, Wang J, Gursoy MC (2017) A deep reinforcement learning based framework for power-efficient resource allocation in edge RANs. In: Proceedings of IEEE international conference on communications, pp 1–6Google Scholar
  21. 21.
    Samuel N, Diskin T, Wiesel A (2017) Deep MIMO detection. In: IEEE 18th Int. Workshop Signal Process. Adv. Wireless Commun., pp. 690–694Google Scholar
  22. 22.
    Ye H, Li GY, Juang BH (2018) Power of deep learning for channel estimation and signal detection in OFDM systems. IEEE Wireless Commun Lett 7(1):114–117CrossRefGoogle Scholar
  23. 23.
    He Y, Zhang Z, Yu FR, Zhao N, Yin H, Leung VC, Zhang Y (2017) Deep-reinforcement-learning-based optimization for cache-enabled opportunistic interference alignment wireless networks. IEEE Trans Veh Technol 66 (11):10433–10445CrossRefGoogle Scholar
  24. 24.
    Zhong C, Gursoy MC, Velipasalar S (2018) A deep reinforcement learning-based framework for content caching. In: IEEE 52nd Annual Conference on Information Sciences and Systems (CISS), pp. 1–6Google Scholar
  25. 25.
    He Y, Yu FR, Zhao N, Leung VC, Yin H (2017) Software-defined networks with mobile edge computing and caching for smart cities: a big data deep reinforcement learning approach. IEEE Commun Mag 55(12):31–37CrossRefGoogle Scholar
  26. 26.
    Min M, Xu D, Xiao L, Tang Y, Wu D (2017) Learning-based computation offloading for IoT devices with energy harvesting. arXiv:1712.08768
  27. 27.
    Chen X, Zhang H, Wu C, Mao S, Ji Y, Bennis M (2018) Performance optimization in mobile-edge computing via deep reinforcement learning. arXiv:1804.00514
  28. 28.
    Huang L, Feng X, Qian LP, Wu Y (2018) Deep reinforcement learning-based task offloading and resource allocation for mobile edge computing. In: 3rd EAI International Conference on Machine Learning and Intelligent Communications, pp 1–10CrossRefGoogle Scholar
  29. 29.
    Huang L, Bi S, Qian LP, Xia Z (2018) Adaptive scheduling in energy harvesting sensor networks for green cities. IEEE Trans Ind Inf 14(4):1575–1584CrossRefGoogle Scholar
  30. 30.
    You C, Huang K, Chae H, Kim B (2017) Energy-Efficient Resource Allocation for Mobile-Edge Computation Offloading. IEEE Trans Wirel Commun 16(3):1397–1411CrossRefGoogle Scholar
  31. 31.
    Zhang H, Liu H, Cheng J, Leung MCV (2018) Downlink Energy Efficiency of Power Allocation and Wireless Backhaul Bandwidth Allocation in Heterogeneous Small Cell Networks. IEEE Trans Commun 66(4):1705–1716CrossRefGoogle Scholar
  32. 32.
    Lin LJ (1993) Reinforcement learning for robots using neural networks, Carnegie-Mellon Univ Pittsburgh PA School of Computer Science, Technical ReportGoogle Scholar
  33. 33.
    Horgan D, Quan J, Budden D, Barth-Maron G, Hessel M, van Hasselt H, Silver D (2018) Distributed prioritized experience replay. arXiv:1803.00933
  34. 34.
    Loshchilov I, Hutter F (2015) Online batch selection for faster training of neural networks. arXiv:1511.06343
  35. 35.
    Alain G, Lamb A, Sankar C, Courville A, Bengio Y (2015) Variance reduction in SGD by distributed importance sampling. arXiv:1511.06481
  36. 36.
    Schaul T, Quan J, Antonoglou I, Silver D (2016) Prioritized experience replay. In International conference on learning representations (ICLR)Google Scholar
  37. 37.
    Abadi M, Agarwal A, Barham P, Brevdo E, Chen Z, Citro C, Corrado GS, Davis A, Dean J, Devin M, Ghemawat S (2016) Tensorflow: large-scale machine learning on heterogeneous distributed systems. arXiv:1603.04467

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.College of Information EngineeringZhejiang University of TechnologyHangzhouChina

Personalised recommendations