Skip to main content

Advertisement

SpringerLink
Log in

Efficient Data Offloading Using Markovian Decision on State Reward Action in Edge Computing

  • Research
  • Published:
Journal of Grid Computing Aims and scope Submit manuscript

Abstract

Efficient planning of Task offloading in edge computing for the Internet of Things (IoT) can increase latency issues and power use. In this research, we propose the task offloading issue as a joint decision-making problem for cost reductions, integrating computation delay, and handling power usage, with the goal of reducing the cost of resources required for task offloading. In this paper, a Markovian Decision Process with Deep Q-Learning (MD-DQN) is used for multi-level task offloading. The Deep Q-learning algorithm greatly enhances the accuracy and handles the task offloading decision process in mobile and edge devices. It anticipates the load on the edge server in real-time. The Markovian Decision Process reduces data unloading while assisting in decision-making. As a result, the task's reaction time will be further reduced, and the system's offloading efficiency will be improved. Employing metrics for computation time and rate power efficacy, offloading ratio, and scheduling faults, the proposed method's performance is compared. The outcomes of the experiments indicate that this proposed MD-DQN method enhances both energy efficiency and computation speed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Data Availability

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

References

  1. Tu, Y., Chen, H., Yan, L., Zhou, X.: Task offloading based on LSTM prediction and deep reinforcement learning for efficient edge computing in IoT. Futur. Internet. 14(2), 30 (2022)

    Article  Google Scholar 

  2. Chen, X., Liu, G.: Federated deep reinforcement learning-based task offloading and resource allocation for smart cities in a mobile edge network. Sensors 22(13), 4738 (2022)

    Article  Google Scholar 

  3. Abbas, N., Zhang, Y., Taherkordi, A., Skeie, T.: Mobile edge computing: A survey. IEEE Internet Things J. 5(1), 450–465 (2017)

    Article  Google Scholar 

  4. Lin, L., Liao, X., Jin, H. and Li, P.,:Computation offloading toward edge computing. In: Proceedings of the IEEE 107(8), 1584–1607 (2019)

  5. Cao, B., Zhao, J., Lv, Z., Yang, P.: Diversified Personalized Recommendation Optimization Based on Mobile Data. IEEE Trans. Intell. Transp. Syst. 22(4), 2133–2139 (2021)

    Article  Google Scholar 

  6. Thai, M.T., Lin, Y.D., Lai, Y.C., Chien, H.T.: Workload and Capacity Optimization for Cloud-Edge Computing Systems with Vertical and Horizontal Offloading. IEEE Trans. Netw. Serv. Manag 17, 227–238 (2020). (IEEE)

    Article  Google Scholar 

  7. Cui, L., Xu, C., Yang, S., Huang, J.Z., Li, J., Wang, X., Ming, Z., Lu, N.: Joint Optimization of Energy Consumption and Latency in Mobile Edge Computing for Internet of Things. IEEE Internet Things J 6, 4791–4803 (2018)

    Article  Google Scholar 

  8. Sonmez, C., Tunca, C., Ozgovde, A., Ersoy, C.: Machine Learning-Based Workload Orchestrator for Vehicular Edge Computing. IEEE Trans. Intell. Transp. Syst 22, 2239–2251 (2021)

    Article  Google Scholar 

  9. Shu, C., Zhao, Z., Han, Y., Min, G., Duan, H.: Multi-User Offloading for Edge Computing Networks: A Dependency-Aware and Latency-Optimal Approach. IEEE Internet Things J. 7, 1678–1689 (2020). (IEEE)

    Article  Google Scholar 

  10. Guo, H., Liu, J.: Collaborative Computation Offloading for Multiaccess Edge Computing Over Fiber-Wireless Networks. IEEE Trans Veh Technol 67, 4514–4526 (2018)

    Article  Google Scholar 

  11. Ma, Q., Xu, S.: Intentional delay can benefit the consensus of second-order multi-agent systems. Automatica 147, 110750 (2023)

    Article  MathSciNet  MATH  Google Scholar 

  12. Zhong, T., Wang, W., Lu, S., Dong, X., Yang, B.: RMCHN: A Residual Modular Cascaded Heterogeneous Network for Noise Suppression in DAS-VSP Records. IEEE Geosci. Remote Sens. Lett. 20, 1–5 (2023)

  13. Ni, Q., Guo, J., Wu, W., Wang, H., Wu, J.: Continuous Influence-Based Community Partition for Social Networks. IEEE Trans. Netw. Sci. Eng. 9(3), 1187–1197 (2022)

    Article  MathSciNet  Google Scholar 

  14. Vu, T.T., Huynh, N.V., Hoang, D.T., Nguyen, D.N., Dutkiewicz, E.: Offloading Energy Efficiency with Delay Constraint for Cooperative Mobile Edge Computing Networks. In Proceedings of the 2018 IEEE Global Communications Conference (GLOBECOM), Abu Dhabi, United Arab Emirates, 9–13 December, pp. 1–6(2018).

  15. Zhang, J., Tang, Y., Wang, H., Xu, K.: ASRO-DIO: Active subspace random optimization based depth inertial odometry. IEEE Trans. Robot. 39(2), 1496–1508 (2023)

  16. Alqerm, I., Pan, J.: DeepEdge: A New QoE-Based Resource Allocation Framework Using Deep Reinforcement Learning for Future Heterogeneous Edge-IoT Applications. IEEE Trans. Netw. Serv. Manag 18, 3942–3954 (2021)

    Article  Google Scholar 

  17. Han, Y., Xu, X., Zhao, Y., Wang, X., Chen, Z., Liu, J.: Impact of consumer preference on the decision-making of prefabricated building developers. J. Civ. Eng. Manag. 28(3), 166–176 (2022)

    Article  Google Scholar 

  18. Cao, B., Yan, Y., Wang, Y., Liu, X., Lin, J.C., Sangaiah, A.K.,... Lv, Z.: A multiobjective intelligent decision-making method for multistage placement of PMU in power grid enterprises. IEEE Trans. Ind. Inform. (2022). https://doi.org/10.1109/TII.2022.3215787

  19. Huang, L., Feng, X., Feng, A., Huang, Y., Qian, L.P.: Distributed deep learning-based offloading for mobile edge computing networks. Mob. Netw. Appl. 27(3), 1123–1130 (2018)

  20. Tang, M., Wong, V.W.S.: Deep reinforcement learning for task offloading in mobile edge computing systems. IEEE Trans. Mob. Comput. 21(6), 1985–1997 (2020)

  21. Han, Y., Wang, L., Kang, R.: Influence of consumer preference and government subsidy on prefabricated building developer’s decision-making: a three-stage game model. J. Civ. Eng. Manag. 29(1), 35–49 (2023)

    Article  Google Scholar 

  22. Dhelim, S., Ning, H.S., Farha, F., Chen, L.M., Atzori, L., Daneshmand, M.: IoT-enabled social relationships meet artificial social intelligence. IEEE Internet Things J. 8, 17817–17828 (2021)

    Article  Google Scholar 

  23. Fu, Y., Li, C., Yu, F.R., Luan, T.H., Zhao, P., Liu, S.: A survey of blockchain and intelligent networking for the metaverse. IEEE Internet Things J. 10(4), 3587–3610 (2023)

  24. Li, Y.Q., Wang, X., Gan, X.Y., Jin, H.M., Fu, L.Y., Wang, X.B.: Learning-aided computation offloading for trusted collaborative mobile edge computing. IEEE Trans. Mob. Comput. 19, 2833–2849 (2019)

    Article  Google Scholar 

  25. Apostolopoulos, P.A., Tsiropoulou, E.E., Papavassiliou, S.: Cognitive data offloading in mobile edge computing for internet of things. IEEE Access 8, 55736–55749 (2020)

    Article  Google Scholar 

  26. Duan, J., Duan, G., Cheng, S., Cao, S., Wang, G.: Fixed-time time-varying output formation–containment control of heterogeneous general multi-agent systems. ISA Trans. (2023). https://doi.org/10.1016/j.isatra.2023.01.008

  27. Chen, J., Xing, H., Xiao, Z., Xu, L., Tao, T.: A DRL Agent for Jointly Optimizing Computation Offloading and Resource Allocation in MEC. IEEE Internet Things J. 8, 17508–17524 (2021)

    Article  Google Scholar 

  28. Xie, X., Tian, Y., Wei, G.: Deduction of sudden rainstorm scenarios: integrating decision makers’ emotions, dynamic Bayesian network and DS evidence theory. Nat. Hazards. 45, 5303435 (2022)

  29. Li, R., Yu, N., Wang, X., Liu, Y., Cai, Z.,... Wang, E.: Model-based synthetic geoelectric sampling for magnetotelluric inversion with deep neural networks. IEEE Trans. Geosci. Remote Sens. 60, 1–14 (2022)

  30. Qin, X., Liu, Z., Liu, Y., Liu, S., Yang, B., Yin, L., Liu, M., Zheng, W.: User OCEAN Personality Model Construction Method Using a BP Neural Network. Electronics 11(19), 3022 (2022)

    Article  Google Scholar 

  31. Du, Y., Qin, B., Zhao, C., Zhu, Y., Cao, J.,... Ji, Y.: A novel spatio-temporal synchronization method of roadside asynchronous MMW radar-camera for sensor fusion. IEEE Trans. Intell. Transp. Syst. 23(11), 1–12, (2021)

  32. Li, Y., Che, P., Liu, C., Wu, D., Du, Y.: Cross-scene pavement distress detection by a novel transfer learning framework. Comput. Aided Civ. Infrastruct. Eng. 36(11), 1398–1415 (2021)

    Article  Google Scholar 

  33. Attiya, I., Abd, E.M., Abualigah, L., Nguyen, T.N., Abd, E., Ahmed, A.: An improved hybrid swarm intelligence for scheduling IoT application tasks in the cloud. IEEE Trans. Ind. Inform 18, 6264–6272 (2022)

    Article  Google Scholar 

  34. Ning, Z.L., Wang, X.J., Rodrigues, J.J., Xia, F.: Joint computation offloading, power allocation, and channel assignment for 5G-enabled traffic management systems. IEEE Trans. Ind. Inform 15, 3058–3067 (2019)

    Article  Google Scholar 

Download references

Funding

This research received no specific grant from any funding agency.

Author information

Authors and Affiliations

  1. China Institute for Radiation Protection (CIRP), Taiyuan, 030006, Shanxi, China

    Mingye Li & Huan Guo

  2. North University of China (NUC), Data Science And Technology, Taiyuan, 030051, Shanxi, China

    Mingye Li & Haiwei Lei

  3. Institute of IR4.0, Universiti Kebangsaan Malaysia, 43600, Bangi, Malaysia

    Riza Sulaiman

  4. Department of Mathematics, College of Science & Arts, King Abdul Aziz University, Rabigh, Saudi Arabia

    Wejdan Deebani & Meshal Shutaywi

Authors
  1. Mingye Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Haiwei Lei
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Huan Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Riza Sulaiman
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Wejdan Deebani
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Meshal Shutaywi
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

M.L: Conceptualization, Methodology, Formal analysis, Supervision, Writing - original draft, Writing - review & editing.H.L: Writing - original draft, Writing - review & editing.H.G: Investigation, Data Curation, Validation, Resources, Writing - review & editing.R.S: Project administration, Investigation, Writing - review & editing.W..D: Writing - original draft, Writing - review & editing.M.S: Project administration, Investigation, Writing - review & editing.

Corresponding author

Correspondence to Mingye Li.

Ethics declarations

Ethics Approval and Consent to Participate

Not applicable.

Consent for Publication

As per journal norms.

Competing Interests

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, M., Lei, H., Guo, H. et al. Efficient Data Offloading Using Markovian Decision on State Reward Action in Edge Computing. J Grid Computing 21, 25 (2023). https://doi.org/10.1007/s10723-023-09659-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10723-023-09659-w

Keywords

Search

Navigation