Online Learning and Optimization for Computation Offloading in D2D Edge Computing and Networks
- 66 Downloads
This paper introduces a framework of device-to-device edge computing and networks (D2D-ECN), a new paradigm for computation offloading and data processing with a group of resource-rich devices towards collaborative optimization between communication and computation. However, the computation process of task intensive applications would be interrupted when capacity-limited battery energy run out. In order to tackle this issue, the D2D-ECN with energy harvesting technology is applied to provide a green computation network and guarantee service continuity. Specifically, we design a reinforcement learning framework in a point-to-point offloading system to overcome challenges of the dynamic nature and uncertainty of renewable energy, channel state and task generation rates. Furthermore, to cope with high-dimensionality and continuous-valued action of the offloading system with multiple cooperating devices, we propose an online approach based on Lyapunov optimization for computation offloading and resource management without priori energy and network information. Numerical results demonstrate that our proposed scheme can reduce system operation cost with low task execution time in D2D-ECN.
KeywordsD2D-ECN Energy harvesting Computation offloading Resource management Reinforcement learning Lyapunpv optimization
This work is supported by the joint fund of the Ministry of Education of China and China Mobile (MCM 20160304), the Fundamental Research Funds for the Central Universities, China (ZYGX2016Z011), and EU H2020 Project COSAFE (MSCA-RISE-2018-824019).
- 2.ETSI ISG (2015) Mobile edge computing a key technology towards 5G. White Paper 11:1–16Google Scholar
- 9.Chen X, Pu L, Gao L, Wu W, Wu D (2017) Exploiting massive D2D collaboration for energy-efficient mobile edge computing. IEEE Commun Mag 24(4):64–71Google Scholar
- 10.Ti N, Le L (2017) Computation offloading leveraging computing resources from edge cloud and mobile peers. In: Proceedings of the IEEE Int. Commun. Confe (ICC)Google Scholar
- 12.Garlatova M, Wallwater A, Zussman G (2011) Networking low-power energy harvesting devices: measurements and algorithms. In: Proceedings of the IEEE Conf. Comput. Commun. (INFOCOM), pp 1602–1610Google Scholar
- 20.Miettinen AP, Nurminen JK (2010) Energy efficiency of mobile clients in cloud computing. In: Proceedings of the USENIX Conf. Hot Topics Cloud Comput. (HotCloud), Boston, MA, USA, pp 1–7Google Scholar
- 24.Altman E (1999) Constrained Markov Decision Process Chapman and Hall/CRCGoogle Scholar
- 25.Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. MIT Press, CambridgeGoogle Scholar
- 27.Neely MJ (2010) Stochastic network optimization with application to communication and queueing systems, San rafael, CA USA: Morgan claypoolGoogle Scholar
- 30.Mitchell TM (1997) Machine learning McGraw-HillGoogle Scholar