Skip to main content

Advertisement

SpringerLink
Log in

On the Joint Design of Microservice Deployment and Routing in Cloud Data Centers

  • Research
  • Published:
Journal of Grid Computing Aims and scope Submit manuscript

Abstract

In recent years, internet enterprises have transitioned from traditional monolithic service to microservice architecture to better meet evolving business requirements. However, it also brings great challenges to the resource management of service providers. Existing research has not fully considered the request characteristics of internet application scenarios. Some studies apply traditional task scheduling models and strategies to microservice scheduling scenarios, while others optimize microservice deployment and request routing separately. In this paper, we propose a microservice instance deployment algorithm based on genetic and local search, and a request routing algorithm based on probabilistic forwarding. The service graph with complex dependencies is decomposed into multiple service chains, and the open Jackson queuing network is applied to analyze the performance of the microservice system. Data evaluation results demonstrate that our scheme significantly outperforms the benchmark strategy. Our algorithm has reduced the average response latency by 37%-67% and enhanced request success rate by 8%-115% compared to other baseline algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Data Availability

No datasets were generated or analysed during the current study.

References

  1. Al-Debagy, O., Martinek, P.: A comparative review of microservices and monolithic architectures. 000149–000154 (2018). https://doi.org/10.1109/CINTI.2018.8928192

  2. Velepucha, V., Flores, P.: Monoliths to microservices - migration problems and challenges: A sms 135–142 (2021). https://doi.org/10.1109/ICI2ST51859.2021.00027

  3. H Zhou, Q.L. M Chen, et al.: Overload control for scaling wechat microservices. Proceedings of the ACM Symposium on Cloud Computing 149–161 (2018). https://doi.org/10.1145/3267809.3267823

  4. T, M.: Adopting microservices at netflix: Lessons for architectural design. https://www.nginx.com/blog/adopting-mic-roservices-at-netflix-lessons-for-team-and-process-design/

  5. Luo, S., Xu, H., Lu, C., Ye, K., Xu, G., Zhang, L., Ding, Y., He, J., Xu, C.: Characterizing microservice dependency and performance: Alibaba trace analysis. Proceedings of the ACM Symposium on Cloud Computing 412–426 (2021). https://doi.org/10.1145/3472883.3487003

  6. Hu, M., Luo, J., Wang, Y., Lukasiewycz, M., Zeng, Z.: Holistic scheduling of real-time applications in time-triggered in-vehicle networks. IEEE Trans. Industrial Inform. 10(3), 1817–1828 (2014). https://doi.org/10.1109/TII.2014.2327389

    Article  Google Scholar 

  7. Lakhan, A., Memon, M.S., Elhoseny, M., Mohammed, M.A., Qabulio, M., Abdel-Basset, M., et al.: Cost-efficient mobility offloading and task scheduling for microservices iovt applications in container-based fog cloud network. Cluster Comput. 25(3), 2061–2083 (2022). https://doi.org/10.1007/s10586-021-03333-0

    Article  Google Scholar 

  8. Zhao, X., Huang, C.: Microservice based computational offloading framework and cost efficient task scheduling algorithm in heterogeneous fog cloud network. IEEE Access 8, 56680–56694 (2020). https://doi.org/10.1109/ACCESS.2020.2981860

    Article  Google Scholar 

  9. Hu, M., Veeravalli, B.: Requirement-aware scheduling of bag-of-tasks applications on grids with dynamic resilience. IEEE Trans. Comput. 62(10), 2108–2114 (2013). https://doi.org/10.1109/TC.2012.164

    Article  MathSciNet  Google Scholar 

  10. Xu, B., Hu, Y., Hu, M., Liu, F., Peng, K., Liu, L.: Iterative dynamic critical path scheduling: An efficient technique for offloading task graphs in mobile edge computing. Appl. Sci. 12(6) (2022). https://doi.org/10.3390/app12063189

  11. Hu, M., Luo, J., Wang, Y., Veeravalli, B.: Adaptive scheduling of task graphs with dynamic resilience. IEEE Trans. Comput. 66(1), 17–23 (2017). https://doi.org/10.1109/TC.2016.2574349

    Article  MathSciNet  Google Scholar 

  12. Yu, R., Kilari, V.T., Xue, G., Yang, D.: Load balancing for interdependent iot microservices. In: IEEE INFOCOM 2019 - IEEE Conference on Computer Communications, pp. 298–306 (2019). https://doi.org/10.1109/INFOCOM.2019.8737450

  13. Hu, Y., Wang, H., Wang, L., Hu, M., Peng, K., Veeravalli, B.: Joint deployment and request routing for microservice call graphs in data centers. IEEE Trans Parallel Distributed Syst. 34(11), 2994–3011 (2023). https://doi.org/10.1109/TPDS.2023.3311767

    Article  Google Scholar 

  14. Zheng, T., Zheng, X., Zhang, Y., Deng, Y., Dong, E., Zhang, R., Liu, X.: Smartvm: a sla-aware microservice deployment framework. World Wide Web 22(1), 275–293 (2019). https://doi.org/10.1007/s11280-018-0562-5

    Article  Google Scholar 

  15. Peng, K.,Wang, L., He, J., Cai, C., Hu, M.: Joint optimization of service deployment and request routing for microservices in mobile edge computing. IEEE Trans. Serv. Comput. 1–13 (2024). https://doi.org/10.1109/TSC.2024.3349408

  16. Rosenwein, M.B.: Discrete location theory. Networks: An Int. J. 24(2), 124–125 (1994). https://doi.org/10.1057/jors.1991.208

  17. Vance, P.H.: Knapsack problems: Algorithms and computer implementations (s. martello and p. toth). SIAM Rev. 35(4), 684–685 (1993). https://doi.org/10.1057/jors.1991.208

  18. Noor, A., Jha, D.N., Mitra, K., Jayaraman, P.P., Souza, A., Ranjan, R., Dustdar, S.: A framework for monitoring microservice-oriented cloud applications in heterogeneous virtualization environments. In: 2019 IEEE 12th International Conference on Cloud Computing (CLOUD), pp. 156–163 (2019). https://doi.org/10.1109/CLOUD.2019.00035

  19. Wang, Y., Shi, W., Hu, M.: Virtual servers comigration for mobile accesses: Online versus offline. IEEE Trans. Mobile Comput. 14(12), 2576–2589 (2015). https://doi.org/10.1109/TMC.2015.2404791

    Article  Google Scholar 

  20. Fard, H.M., Prodan, R., Wolf, F.: Dynamic multi-objective scheduling of microservices in the cloud. In: 2020 IEEE/ACM 13th International Conference on Utility and Cloud Computing (UCC), pp. 386–393 (2020). https://doi.org/10.1109/UCC48980.2020.00061

  21. Wang, S., Ding, Z., Jiang, C.: Elastic scheduling for microservice applications in clouds. IEEE Trans. Parallel Distributed Syst. 32(1), 98–115 (2021). https://doi.org/10.1109/TPDS.2020.3011979

    Article  Google Scholar 

  22. Zhao, D., Zou, Q., Boshkani Zadeh, M.: A qosaware iot service placement mechanism in fog computing based on open-source development model. J. Grid Comput. 20(2), 12 (2022). https://doi.org/10.1007/s10723-022-09604-3

    Article  Google Scholar 

  23. Sami, H., Mourad, A., El-Hajj, W.: Vehicularobus-as-on-demand-fogs: Resource and context aware deployment of containerized microservices. IEEE/ACM Transactions on Networking 28(2), 778–790 (2020). https://doi.org/10.1109/TNET.2020.2973800

    Article  Google Scholar 

  24. Lv, J., Wei, M., Yu, Y.: A container scheduling strategy based on machine learning in microservice architecture. In: 2019 IEEE International Conference on Services Computing (SCC), pp. 65–71 (2019). https://doi.org/10.1109/SCC.2019.00023

  25. Bao, L., Wu, C., Bu, X., Ren, N., Shen, M.: Performance modeling and workflow scheduling of microservice-based applications in clouds. IEEE Trans. Parallel Distributed Syst. 30(9), 2114–2129 (2019). https://doi.org/10.1109/TPDS.2019.2901467

    Article  Google Scholar 

  26. Mechtri, M., Ghribi, C., Zeghlache, D.: A scalable algorithm for the placement of service function chains. IEEE Trans. Netw. Serv. Manag. 13(3), 533–546 (2016). https://doi.org/10.1109/TNSM.2016.2598068

    Article  Google Scholar 

  27. Luizelli, M.C., Bays, L.R., Buriol, L.S., Barcellos, M.P., Gaspary, L.P.: Piecing together the nfv provisioning puzzle: Efficient placement and chaining of virtual network functions, 98–106 (2015). https://doi.org/10.1109/INM.2015.7140281

  28. Wei, H., Rodriguez, J.S., Garcia, O.N.-T.: Deployment management and topology discovery of microservice applications in the multicloud environment. J. Grid Comput. 19, 1–22 (2021). https://doi.org/10.1007/s10723-021-09539-1

    Article  Google Scholar 

  29. Amiri, A., Zdun, U., Hoorn, A.: Modeling and empirical validation of reliability and performance trade-offs of dynamic routing in serviceand cloud-based architectures. IEEE Trans. Serv. Comput. 15(6), 3372–3386 (2022). https://doi.org/10.1109/TSC.2021.3098178

    Article  Google Scholar 

  30. Hu, M., Luo, J., Wang, Y., Veeravalli, B.: Practical resource provisioning and caching with dynamic resilience for cloud-based content distribution networks. IEEE Trans. Parallel Distributed Syst. 25(8), 2169–2179 (2014). https://doi.org/10.1109/TPDS.2013.287

    Article  Google Scholar 

  31. Cui, J., Chen, P., Yu, G.: A learning-based dynamic load balancing approach for microservice systems in multi-cloud environment. In: 2020 IEEE 26th International Conference on Parallel and Distributed Systems (ICPADS), pp. 334–341 (2020). https://doi.org/10.1109/ICPADS51040.2020.00052

  32. Fan, Q., Yin, H., Jiao, L., Lyu, Y., Huang, H., Zhang, X.: Towards optimal request mapping and response routing for content delivery networks. IEEE Trans. Serv. Comput. 14(2), 606–613 (2021). https://doi.org/10.1109/TSC.2018.2796567

    Article  Google Scholar 

  33. Annie Poornima Princess, G., Radhamani, A.: A hybrid meta-heuristic for optimal load balancing in cloud computing. J. Grid Comput. 19(2), 21 (2021). https://doi.org/10.1007/s10723-021-09560-4

  34. Liu, Z., Long, C., Lu, X., Hu, Z., Zhang, J., Wang, Y.: Which channel to ask my question?: Personalized customer service request stream routing using deep reinforcement learning. IEEE Access 7, 107744–107756 (2019). https://doi.org/10.1109/ACCESS.2019.2932047

    Article  Google Scholar 

  35. Pei, J., Hong, P., Xue, K., Li, D.: Resource aware routing for service function chains in sdn and nfv-enabled network. IEEE Trans. Serv. Comput. 14(4), 985–997 (2021). https://doi.org/10.1109/TSC.2018.2849712

    Article  Google Scholar 

  36. Chen, X., Bi, Y., Chen, X., Zhao, H., Cheng, N., Li, F., Cheng, W.: Dynamic service migration and request routing for microservice in multicell mobile-edge computing. IEEE Int. Things J. 9(15), 13126–13143 (2022). https://doi.org/10.1109/JIOT.2022.3140183

    Article  Google Scholar 

  37. Lim, K., Bang, Y., Sung, J., Rhee, J.-K.K.: Joint optimization of cache server deployment and request routing with cooperative content replication. In: 2014 IEEE International Conference on Communications (ICC), pp. 1790–1795 (2014). https://doi.org/10.1109/ICC.2014.6883582

  38. Yu, Y., Yang, J., Guo, C., Zheng, H., He, J.: Joint optimization of service request routing and instance placement in the microservice system. J. Netw. Computer Appl. 147, 102441 (2019). https://doi.org/10.1016/j.jnca.2019.102441

    Article  Google Scholar 

  39. Zhang, Q., Xiao, Y., Liu, F., Lui, J.C.S., Guo, J., Wang, T.: Joint optimization of chain placement and request scheduling for network function virtualization. In: 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS), pp. 731–741 (2017). https://doi.org/10.1109/ICDCS.2017.232

  40. Benson, T., Akella, A., Maltz, D.A.: Network traffic characteristics of data centers in the wild 267–280 (2010). https://doi.org/10.1145/1879141.1879175

  41. Xia, W., Zhao, P., Wen, Y., Xie, H.: A survey on data center networking (dcn): Infrastructure and operations. IEEE Commun. Surv. & Tutorials 19(1), 640–656 (2016). https://doi.org/10.1109/COMST.2016.2626784

    Article  Google Scholar 

  42. Dawoud, W., Takouna, I., Meinel, C.: Elastic virtual machine for fine-grained cloud resource provisioning 11–25 (2012). https://doi.org/10.1007/978-3-642-29219-4_2

  43. Pacifici, G., Spreitzer, M., Tantawi, A.N., Youssef, A.: Performance management for cluster-based web services. IEEE J. Selected Areas Commun. 23(12), 2333–2343 (2005). https://doi.org/10.1109/JSAC.2005.857208

    Article  Google Scholar 

  44. Niu, Y., Liu, F., Li, Z.: Load balancing across microservices. In: IEEE INFOCOM 2018 - IEEE Conference on Computer Communications, pp 198–206 (2018). https://doi.org/10.1109/INFOCOM.2018.8486300

  45. Fu, T.Z.J., Ding, J., Ma, R.T.B., Winslett, M., Yang, Y., Zhang, Z.: Drs: Auto-scaling for realtime stream analytics. IEEE/ACM Trans. Netw. 25(6), 3338–3352 (2017). https://doi.org/10.1109/TNET.2017.2741969

    Article  Google Scholar 

  46. Karel, A.E.J.: Local search in combinatorial optimization (2003)

  47. Li, H., Tang, B., Xu, W., Guo, F., Zhang, X.: Application deployment in mobile edge computing environment based on microservice chain 531–536 (2022). https://doi.org/10.1109/CSCWD54268.2022.9776307

  48. Mohan, A., Kaseb, A.S., Lu, Y.-H., Hacker, T.J.: Adaptive resource management for analyzing video streams from globally distributed network cameras. IEEE Trans. Cloud Comput. 9(1), 40–53 (2021). https://doi.org/10.1109/TCC.2018.2836907

    Article  Google Scholar 

Download references

Funding

This work is supported by the National Key Research and Development Program (2022ZD0117104) and the National Natural Science Foundation of China (62171189, 62272183). This work is also supported by Key Research and Development Program of Hubei Province, China (2021BAA026, 2022BAA038, 2023BAB074).

Author information

Authors and Affiliations

  1. Hubei Key Laboratory of Smart Internet Technology, School of Electronic Information and Communications, Huazhong University of Science and Technology, Wuhan, 430074, People’s Republic of China

    Bo Xu, Jialu Guo, Fangling Ma, Menglan Hu, Wei Liu & Kai Peng

Authors
  1. Bo Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jialu Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fangling Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Menglan Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Wei Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Kai Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Xu and Guo wrote the main manuscript text and designed the method model, Ma and Hu carried out algorithm design and experiments, as well as Liu and Peng prepared all the figures and revised the manuscript.

Corresponding author

Correspondence to Kai Peng.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xu, B., Guo, J., Ma, F. et al. On the Joint Design of Microservice Deployment and Routing in Cloud Data Centers. J Grid Computing 22, 42 (2024). https://doi.org/10.1007/s10723-024-09759-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10723-024-09759-1

Keywords

Search

Navigation