Skip to main content

Advertisement

SpringerLink
Log in

Toward intelligent cooperation at the edge: improving the QoS of workflow scheduling with the competitive cooperation of edge servers

  • Original Paper
  • Published:
Wireless Networks Aims and scope Submit manuscript

Abstract

Advances in big data and Internet of Things devices have brought novel service modes, such as smart cities and intelligent transportation, to daily life. With the widespread deployment of smart terminals comes an exponentially increasing amount of data, which, causes conflict due to the intensive resource demand and limited computation capacity. To manage this conflict, edge computing has been introduced as an auxiliary technique to cloud computing. However, the emerging computation-intensive service chains bring high resource demands that may exceed the computation capability of a single edge server. Simply offloading them to cloud servers is hardly time saving and is challenging for typical edge-cloud schemes. In this paper, we address the challenge of coordinating the workflow scheduler from multiple users in a partially observable environment. We first partition the workflow by leveraging graph theory to split the component tasks into clusters based on their dependency constraints. We further model the possible contention on edge servers among multiple users as a Markov game and propose a multiagent reinforcement learning-based edge server coordination algorithm named partially observable multiagent workflow scheduler (POMAWS) as the solution. With fine-trained agents, the proposed scheme can intelligently activate nearby edge nodes to form a temporal workgroup and manage contention when it occurs. The numerical results validate the feasibility of our proposed scheme, as its performance exceeds typical cloud computing and traditional clustering schemes with an improved QoS in terms of processing delay.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data availability

The data that support the findings of this study are available from the corresponding author, Z. Zhang, upon reasonable request.

References

  1. Kim, T. H., Ramos, C., & Mohammed, S. (2017). Smart city and IoT. Future Generation Computer Systems, 76, 159–162. https://doi.org/10.1016/j.future.2017.03.034

    Article  Google Scholar 

  2. Patel, P., Narmawala, Z., & Thakkar, A. (2019). A survey on intelligent transportation system using internet of things. Emerging Research in Computing, Information, Communication and Applications: ERCICA, 2018(1), 231–240. https://doi.org/10.1007/978-981-13-5953-8_20

    Article  Google Scholar 

  3. Rodriguez, M. A., & Buyya, R. (2017). A taxonomy and survey on scheduling algorithms for scientific workflows in IaaS cloud computing environments. Concurrency and Computation: Practice and Experience, 29(8), 1–23. https://doi.org/10.1002/cpe.4041

    Article  Google Scholar 

  4. Chitra, S. (2020). Compensatory Aggregation based Failure Aware Cloud Workflow Scheduling. In 2020 2nd International Conference on Innovative Mechanisms for Industry Applications (ICIMIA), IEEE. https://doi.org/10.1109/ICIMIA48430.2020.9074964

  5. Li, H., Ota, K., & Dong, M. (2018). Learning IoT in edge: Deep learning for the Internet of Things with edge computing. IEEE Network, 32(1), 96–101. https://doi.org/10.1109/MNET.2018.1700202

    Article  Google Scholar 

  6. Guo, H., Liu, J., & Lv, J. (2019). Toward intelligent task offloading at the edge. IEEE Network, 34(2), 128–134. https://doi.org/10.1109/MNET.001.1900200

    Article  Google Scholar 

  7. Liu, S. W., Kong, L. M., Ren, K. J., Song, J. Q., Deng, K. F., & Leng, H. Z. (2011). A two-step data placement and task scheduling strategy for optimizing scientific workflow performance on cloud computing platform. Jisuanji Xuebao (Chinese Journal of Computers), 34(11), 2121–2130. https://doi.org/10.3724/SP.J.1016.2011.02121

    Article  Google Scholar 

  8. Tanaka, M., & Tatebe, O. (2012). Workflow scheduling to minimize data movement using multi-constraint graph partitioning. In 2012 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (ccgrid 2012), IEEE. https://doi.org/10.1109/CCGrid.2012.134

  9. Yang, L., Cao, J., Yuan, Y., Li, T., Han, A., & Chan, A. (2013). A framework for partitioning and execution of data stream applications in mobile cloud computing. ACM SIGMETRICS Performance Evaluation Review, 40(4), 23–32. https://doi.org/10.1145/2479942.2479946

    Article  Google Scholar 

  10. Ning, H., Li, Y., Shi, F., & Yang, L. T. (2020). Heterogeneous edge computing open platforms and tools for internet of things. Future Generation Computer Systems, 106, 67–76. https://doi.org/10.1016/j.future.2019.12.036

    Article  Google Scholar 

  11. Lauer, M. (2000). An algorithm for distributed reinforcement learning in cooperative multiagent systems. In Proc. 17th International Conf. on Machine Learning

  12. Omidshafiei, S., Pazis, J., Amato, C., How, J. P., & Vian, J. (2017). Deep decentralized multi-task multi-agent reinforcement learning under partial observability. In International Conference on Machine Learning

  13. Wang, J., Hong, Y., Wang, J., Xu, J., Tang, Y., Han, Q. L., & Kurths, J. (2022). Cooperative and competitive multi-agent systems: From optimization to games. IEEE/CAA Journal of Automatica Sinica, 9(5), 763–783. https://doi.org/10.1109/JAS.2022.105506

    Article  Google Scholar 

  14. Zhang, Y., Di, B., Zheng, Z., Lin, J., & Song, L. (2019). Joint data offloading and resource allocation for multi-cloud heterogeneous mobile edge computing using multi-agent reinforcement learning. In 2019 IEEE Global Communications Conference (GLOBECOM), IEEE. https://doi.org/10.1109/GLOBECOM38437.2019.9013596

  15. Lowe, R., Wu, Y. I., Tamar, A., Harb, J., Pieter Abbeel, O., & Mordatch, I. (2017). Multi-agent actor-critic for mixed cooperative-competitive environments. Advances in neural information processing systems 30 (NIPS 2017)

  16. Jayanetti, A., Halgamuge, S., & Buyya, R. (2022). Deep reinforcement learning for energy and time optimized scheduling of precedence-constrained tasks in edge–cloud computing environments. Future Generation Computer Systems, 137, 14–30. https://doi.org/10.1016/j.future.2022.06.012

    Article  Google Scholar 

  17. Zhang, C., Song, W., Cao, Z., Zhang, J., Tan, P. S., & Chi, X. (2020). Learning to dispatch for job shop scheduling via deep reinforcement learning. Advances in Neural Information Processing Systems, 33, 1621–1632.

    Google Scholar 

  18. Tuli, S., Mahmud, R., Tuli, S., & Buyya, R. (2019). Fogbus: A blockchain-based lightweight framework for edge and fog computing. Journal of Systems and Software, 154, 22–36. https://doi.org/10.1016/j.jss.2019.04.050

    Article  Google Scholar 

  19. Tuli, S., Casale, G., & Jennings, N. R. (2021). Mcds: Ai augmented workflow scheduling in mobile edge cloud computing systems. IEEE Transactions on Parallel and Distributed Systems, 33(11), 2794–2807. https://doi.org/10.1109/TPDS.2021.3135907

    Article  Google Scholar 

  20. Zhou, Y., Li, X., Luo, J., Yuan, M., Zeng, J., & Yao, J. (2022). Learning to optimize dag scheduling in heterogeneous environment. In 2022 23rd IEEE International Conference on Mobile Data Management (MDM), IEEE. https://doi.org/10.1109/MDM55031.2022.00040

  21. Sun, P., Guo, Z., Wang, J., Li, J., Lan, J., & Hu, Y. (2021). Deepweave: Accelerating job completion time with deep reinforcement learning-based coflow scheduling. In Proceedings of the Twenty-Ninth International Conference on International Joint Conferences on Artificial Intelligence, (pp. 3314–3320)

  22. Wu, H., Knottenbelt, W., Wolter, K., & Sun, Y. (2016). An optimal offloading partitioning algorithm in mobile cloud computing. In Quantitative Evaluation of Systems: 13th International Conference (QEST 2016), 311–328. https://doi.org/10.1007/978-3-319-43425-4_21

  23. Wu, Z., & Jiang, L. (2019). Application offloading algorithm based on multi-edge node collaboration. Journal of Nanjing University of Posts and Telecommunications (Natural Science Edition), 39(4), 96–102.

    Google Scholar 

  24. Stanton, I., & Kliot, G. (2012). Streaming graph partitioning for large distributed graphs. In Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining, (pp. 1222–1230). https://doi.org/10.1145/2339530.2339722

  25. Chandru, V., & Rao, M. (1998). Integer programming. IIM Bangalore Research Paper, (p. 110)

  26. Moura, J., & Hutchison, D. (2018). Game theory for multi-access edge computing: Survey, use cases, and future trends. IEEE Communications Surveys & Tutorials, 21(1), 260–288. https://doi.org/10.1109/COMST.2018.2863030

    Article  Google Scholar 

  27. Schwartz, H. M. (2014). Multi-agent machine learning: A reinforcement approach. John Wiley & Sons. https://doi.org/10.1002/9781118884614

    Article  MATH  Google Scholar 

  28. Simões, D., Lau, N., & Reis, L. P. (2020). Multi-agent actor centralized-critic with communication. Neurocomputing, 390, 40–56. https://doi.org/10.1016/j.neucom.2020.01.079

    Article  Google Scholar 

  29. Sukhbaatar, S., Lin, Z., Kostrikov, I., Synnaeve, G., Szlam, A., & Fergus, R. (2018). Intrinsic motivation and automatic curricula via asymmetric self-play. In 6th International Conference on Learning Representations (ICLR 2018)

  30. Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., Van Den Driessche, G., & Hassabis, D. (2016). Mastering the game of Go with deep neural networks and tree search. Nature, 529(7587), 484–489. https://doi.org/10.1038/nature16961

    Article  Google Scholar 

  31. Nowé, A., Vrancx, P., & De Hauwere, Y. M. (2012). Game theory and multi-agent reinforcement learning. Reinforcement Learning: State-of-the-Art, 2012, 441–470.

    Article  MATH  Google Scholar 

  32. Hu, J., & Wellman, M. P. (1998). Multiagent reinforcement learning: theoretical framework and an algorithm. ICML, 98, 242–250. https://doi.org/10.5555/645527.657296

    Article  Google Scholar 

  33. Hu, J., & Wellman, M. P. (2003). Nash Q-learning for general-sum stochastic games. Journal of machine learning research, 4, 1039–1069.

    MathSciNet  MATH  Google Scholar 

  34. Lin, Y., Gade, S., Sandhu, R., & Liu, J. (2020). Toward resilient multi-agent actor-critic algorithms for distributed reinforcement learning. In 2020 American Control Conference (ACC), IEEE, (pp. 3953–3958). https://doi.org/10.23919/ACC45564.2020.9147381

  35. Chen, X., Tang, S., Lu, Z., Wu, J., Duan, Y., Huang, S. C., & Tang, Q. (2019). iDiSC: A new approach to IoT-data-intensive service components deployment in edge-cloud-hybrid system. IEEE Access, 7, 59172–59184. https://doi.org/10.1109/ACCESS.2019.2915020

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China (NO. 62173026) and the Fundamental Research Funds for the Central Universities under Grant 2022JBZY002.

Author information

Authors and Affiliations

  1. School of Electronic and Information Engineering, Beijing Jiaotong University, Beijing, China

    Kaige Zhu, Zhenjiang Zhang & Feng Sun

  2. Key Laboratory of Communication and Information Systems, Beijing Municipal Commission of Education, Beijing, China

    Kaige Zhu, Zhenjiang Zhang & Feng Sun

  3. Trusted Cloud Computing and Big Data Key Laboratory of Sichuan Province, Chengdu, China

    Zhenjiang Zhang

Authors
  1. Kaige Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhenjiang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Feng Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhenjiang Zhang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhu, K., Zhang, Z. & Sun, F. Toward intelligent cooperation at the edge: improving the QoS of workflow scheduling with the competitive cooperation of edge servers. Wireless Netw (2023). https://doi.org/10.1007/s11276-023-03361-1

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11276-023-03361-1

Keywords

Search

Navigation