Skip to main content
SpringerLink
Log in

On practitioners’ concerns when adopting service mesh frameworks

  • Published:
Empirical Software Engineering Aims and scope Submit manuscript

Abstract

Context

The emerging service mesh architecture tries to simplify microservices by delegating crucial tasks to dedicated infrastructure. However, service mesh introduces new notions and enables complex capabilities such as sidecar proxies that inevitably bring major adoption concerns.

Objective

We investigate the adoption concerns in two dominant open-source service mesh frameworks via a mixed-methods empirical investigation of the past, current and evolution of 5,497 practitioner questions posted on generic and framework-specific question-and-answer fora.

Method

We first mine the topics of questions with the help of Dynamic Topic Modeling (DTM). We identify evolution by applying topic modelling to time periods and aggregating topics into macro-topics. We conduct a qualitative analysis to understand the three major types of questions and to generalize common fix patterns for the extracted error symptoms. We consulted a service mesh domain expert to provide feedback on our findings and discuss implications.

Results

We found that about half of the questions are error-related and mined 18 topics, covering service mesh traffic, infrastructure, security, observability and application. We discovered a drastic decline in traffic-related concerns while finding persisting infrastructure-related concerns and a rise in security and observability concerns. We identified 54 error symptoms from two popular service mesh frameworks and generalized 9 common fix patterns. We found complex symptom-to-fix relationships, yet, surprisingly, minimal configuration changes were able to fix most symptoms.

Conclusion

Providing consistent documentation and practical automation that assists customization of service mesh deployment and functionalities is crucial in the current service mesh domain, given the diversity of discovered intentions, goals and symptoms. Furthermore, there should be more work towards better container orchestration to deploy service mesh frameworks and reliable customization of security and observability service mesh features.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

Data Availability

The datasets generated and analysed during the current study are available from the corresponding author on reasonable request.

Notes

  1. https://www.redhat.com/en/topics/microservices/what-is-a-service-mesh

  2. https://kubernetes.io/

  3. https://www.youtube.com/watch?v=CZ3wIuvmHeM

  4. https://kubernetes.io/blog/2015/06/the-distributed-system-toolkit-patterns/#example-1-sidecar-containers

  5. https://istio.io/latest/docs/ops/deployment/architecture/

  6. https://github.com/envoyproxy/envoy

  7. https://github.com/istio/istio

  8. https://www.cncf.io/wp-content/uploads/2020/11/CNCF_Survey_Report_2020.pdf

  9. https://github.com/hashicorp/consul

  10. https://github.com/linkerd/linkerd2

  11. https://kuma.io/

  12. https://traefik.io/traefik-mesh/

  13. https://cilium.io/

  14. The scripts to collect, process, filter and analyze data are available in the replication package (Chen et al. 2022)

  15. https://cloud.google.com/bigquery

  16. https://www.discourse.org/

  17. https://spring.io/projects/spring-cloud

  18. https://istio.io/latest/docs/ops/deployment

  19. https://www.vaultproject.io/

  20. https://istio.io/latest/blog/2021/ncc-security-assessment/

  21. https://blogs.gartner.com/andrew-lerner/2021/10/11/networking-hype-cycle-2021/

  22. https://discuss.istio.io/t/jwt-policy-does-not-take-affect/141/25

  23. https://kubernetes.io/docs/concepts/overview/working-with-objects/kubernetes-objects/

  24. https://opentelemetry.io/

  25. https://blogs.gartner.com/andrew-lerner/2021/10/11/networking-hype-cycle-2021/

  26. https://youtu.be/Du8ImGRd2TI

References

  • Allamanis M, Sutton C (2013) Why, when, and what: analyzing stack overflow questions by topic, type, and code. In: Proceedings of the 10th Working Conference on Mining Software Repositories (MSR). pp 53–56

  • Alshuqayran N, Ali N, Evans R (2016) A systematic mapping study in microservice architecture. In: Proceedings of the 9th International Conference on Service-Oriented Computing and Applications (SOCA). pp 44–51

  • Aly M, Khomh F, Yacout S (2021) What do practitioners discuss about iot and industry 4.0 related technologies? characterization and identification of iot and industry 4.0 categories in stack overflow discussions. Internet of Things 14:100364

  • Barua A, Thomas SW, Hassan AE (2014) What are developers talking about? an analysis of topics and trends in stack overflow. Empir Softw Eng 19(3):619–654

    Article  Google Scholar 

  • Basili V, Rombach H (1988) The TAME project: Towards improvement-oriented software environments. IEEE Trans Softw Eng 14(6):758–773

    Article  Google Scholar 

  • Ben-Kiki O, Evans C, Ingerson B (2009) Yaml ain’t markup language (yaml™) version 1.1. Working Draft 2008 5:11

  • Bevana N, Kirakowskib J, Maissela J (1991) What is usability? In: Proceedings of the 4th International Conference on Human-Computer Interaction (HCI International). pp 1–5

  • Blei DM, Lafferty JD (2006) Dynamic topic models. In: Proceedings of the 23rd International Conference on Machine Learning (ICML). pp 113–120

  • Blei DM, Ng AY, Jordan MI (2003) Latent dirichlet allocation. The Journal of Machine Learning Research 3:993–1022

    MATH  Google Scholar 

  • Bourne V (2022) Annual apis and integration report - 2022. Tech. rep., Software AG. URL https://www.softwareag.com/en_corporate/resources/asset/ar/integrationapi/apis-integration-microservices-report.html

  • Buck C, Olenberger C, Schweizer A, Völter F, Eymann T (2021) Never trust, always verify: A multivocal literature review on current knowledge and research gaps of zerotrust. Computers & Security 110:102436

    Article  Google Scholar 

  • Burns B, Grant B, Oppenheimer D, Brewer E, Wilkes J (2016) Borg, omega, and kubernetes. Commun ACM 59(5):50–57

    Article  Google Scholar 

  • Cardellini V, Colajanni M, Yu PS (1999) Dynamic load balancing on web-server systems. IEEE Internet Computing 3(3):28–39

    Article  Google Scholar 

  • Chakraborty P, Shahriyar R, Iqbal A, Uddin G (2021) How do developers discuss and support new programming languages in technical q &a site? an empirical study of go, swift, and rust in stack overflow. Inf Softw Technol 137:106603

    Article  Google Scholar 

  • Chen Y, Fernandes E, Adams B, Hassan AE (2022) Replication package of the paper ’on practitioners’ concerns when adopting service mesh frameworks’. URL https://www.dropbox.com/scl/fo/qrwbg9w9941xoowaoxyb0/h?dl=0 &rlkey=lkumrmmc4eez4onu5lnodmz4x

  • Chen Z, Cao Y, Liu Y, Wang H, Xie T, Liu X (2020) A comprehensive study on challenges in deploying deep learning based software. In: Proceedings of the 28th Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (FSE). pp 750–762

  • Cito J, Schermann G, Wittern JE, Leitner P, Zumberi S, Gall HC (2017) An empirical analysis of the docker container ecosystem on github. In: Proceedings of the 14th International Conference on Mining Software Repositories (MSR). pp 323–333

  • De Sanctis M, Muccini H, Vaidhyanathan K (2020) Data-driven adaptation in microservice-based iot architectures. In: Proceedings of the 3rd International Conference on Software Architecture Companion (ICSA-C). pp 59–62

  • Dragoni N, Giallorenzo S, Lafuente AL, Mazzara M, Montesi F, Mustafin R, Safina L (2017) Microservices: yesterday, today, and tomorrow. Present and Ulterior Software Engineering. pp 195–216

  • Esposito C, Castiglione A, Choo KKR (2016) Challenges in delivering software in the cloud as microservices. IEEE Cloud Computing 3(5):10–14

    Article  Google Scholar 

  • Ferreira AP, Sinnott R (2019) A performance evaluation of containers running on managed kubernetes services. In: Proceedings of the 11th International Conference on Cloud Computing Technology and Science (CloudCom). pp 199–208

  • Ganguli M, Ranganath S, Ravisundar S, Layek A, Ilangovan D, Verplanke E (2021) Challenges and opportunities in performance benchmarking of service mesh for the edge. In: Proceedings of the 5th International Conference on Edge Computing (EDGE). pp 78–85

  • Goethals T, Volckaert B, De Turck F (2020) Adaptive fog service placement for real-time topology changes in kubernetes clusters. In: Proceedings of the 10th International Conference on Cloud Computing and Services Science (CLOSER). pp 161–170

  • Hahn DA, Davidson D, Bardas AG (2020) Mismesh: Security issues and challenges in service meshes. In: Proceedings of the 13th International Conference on Security and Privacy in Communication Systems (SecureComm). pp 140–151

  • Hamidi S, Andritsos P, Liaskos S (2014) Constructing adaptive configuration dialogs using crowd data. In: Proceedings of the 29th international conference on Automated software engineering (ASE). pp 485–490

  • Hindle A, Godfrey MW, Holt RC (2009) What’s hot and what’s not: Windowed developer topic analysis. In: Proceedings of the 25th International Conference on Software Maintenance (ICSM). pp 339–348

  • Houmani Z, Balouek-Thomert D, Caron E, Parashar M (2020) Enhancing microservices architectures using data-driven service discovery and qos guarantees. In: Proceedings of the 20th International Symposium on Cluster, Cloud and Internet Computing (CCGRID). pp 290–299

  • Klein M (2017) Lyft’s envoy: Experiences operating a large service mesh. URL https://www.usenix.org/conference/srecon17americas/program/presentation/klein

  • Li B, Peng X, Xiang Q, Wang H, Xie T, Sun J, Liu X (2022) Enjoy your observability: an industrial survey of microservice tracing and analysis. Empir Softw Eng 27(1):1–28

    Article  Google Scholar 

  • Li W, Lemieux Y, Gao J, Zhao Z, Han Y (2019) Service mesh: Challenges, state of the art, and future research opportunities. In: Proceedings of the 13th International Conference on Service-Oriented System Engineering (SOSE). pp 122–1225

  • Lou Y, Chen Z, Cao Y, Hao D, Zhang L (2020) Understanding build issue resolution in practice: symptoms and fix patterns. In: Proceedings of the 28th Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (FSE). pp 617–628

  • McHugh ML (2013) The chi-square test of independence. Biochemia Medica 23(2):143–149

    Article  Google Scholar 

  • Miano S, Bertrone M, Risso F, Tumolo M, Bernal MV (2018) Creating complex network services with ebpf: Experience and lessons learned. In: Proceedings of the 19th International Conference on High Performance Switching and Routing (HPSR). pp 1–8

  • Picoreti R, do Carmo AP, de Queiroz FM, Garcia AS, Vassallo RF, Simeonidou D (2018) Multilevel observability in cloud orchestration. In: Proceedings of the 16th International Conference on Dependable, Autonomic and Secure Computing, 16th International Conference on Pervasive Intelligence and Computing, 4th International Conference on Big Data Intelligence and Computing and Cyber Science and Technology Congress (DASC/PiCom/DataCom/CyberSciTech). pp 776–784

  • Röder M, Both A, Hinneburg A (2015) Exploring the space of topic coherence measures. In: Proceedings of the 9th International Conference on Web Search and Data Mining (WSDM). pp 399–408

  • Rose S, Borchert O, Mitchell S, Connelly S (2020) Zero trust architecture. Tech. rep, National Institute of Standards and Technology

    Book  Google Scholar 

  • Saleh Sedghpour MR, Klein C, Tordsson J (2022) An empirical study of service mesh traffic management policies for microservices. In: Proceedings of the 38th International Conference on Performance Engineering (ICPE). pp 17–27

  • Sayagh M, Kerzazi N, Adams B, Petrillo F (2018) Software configuration engineering in practice interviews, survey, and systematic literature review. IEEE Trans Softw Eng 46(6):646–673

    Article  Google Scholar 

  • Schneider S, Khalili R, Manzoor A, Qarawlus H, Schellenberg R, Karl H, Hecker A (2021) Self-learning multi-objective service coordination using deep reinforcement learning. IEEE Trans Netw Serv Manag 18(3):3829–3842

    Article  Google Scholar 

  • Schneider S, Klenner LD, Karl H (2020) Every node for itself: Fully distributed service coordination. In: Proceedings of the 16th International Conference on Network and Service Management (CNSM). pp 1–9

  • Scoccia GL, Migliarini P, Autili M (2021) Challenges in developing desktop web apps: a study of stack overflow and github. In: Proceedings of the 18th International Conference on Mining Software Repositories (MSR). pp 271–282

  • Sculley D, Holt G, Golovin D, Davydov E, Phillips T, Ebner D, Chaudhary V, Young M, Crespo JF, Dennison D (2015) Hidden technical debt in machine learning systems. Advances in Neural Information Processing Systems 28

  • Sedghpour MRS, Klein C, Tordsson J (2021) Service mesh circuit breaker: From panic button to performance management tool. In: Proceedings of the 1st Workshop on High Availability and Observability of Cloud Systems (HAOC). pp 4–10

  • Shadija D, Rezai M, Hill R (2017) Towards an understanding of microservices. In: Proceedings of the 23rd International Conference on Automation and Computing (ICAC). pp 1–6

  • Squire M (2015) “Should we move to stack overflow?” measuring the utility of social media for developer support. In: Proceedings of the 37th International Conference on Software Engineering (ICSE), vol. 2. pp 219–228

  • Syed S, Spruit M (2017) Full-text or abstract? examining topic coherence scores using latent dirichlet allocation. In: 2017 IEEE International conference on data science and advanced analytics (DSAA). pp 165–174. IEEE

  • Thönes J (2015) Microservices. IEEE Software 32(1):116–116

    Article  Google Scholar 

  • Vayghan LA, Saied MA, Toeroe M, Khendek F (2018) Deploying microservice based applications with kubernetes: Experiments and lessons learned. In: Proceedings of the 11th International Conference on Cloud Computing (CLOUD). pp 970–973

  • Venkatesh PK, Wang S, Zhang F, Zou Y, Hassan AE (2016) What do client developers concern when using web apis? an empirical study on developer forums and stack overflow. In: Proceedings of the 13th International Conference on Web Services (ICWS). pp 131–138

  • Wang W, Godfrey MW (2013) Detecting api usage obstacles: A study of ios and android developer questions. In: Proceedings of the 10thWorking Conference on Mining Software Repositories (MSR). pp 61–64

  • Wohlin C, Runeson P, Höst M, Ohlsson M, Regnell B, Wesslén, A (2012) Experimentation in Software Engineering, 1st edn. Springer Science & Business Media

  • Wojciechowski L, Opasiak K, Latusek J, Wereski M, Morales V, Kim T, Hong M (2021) Netmarks: Network metrics-aware kubernetes scheduler powered by service mesh. In: Proceedings of the 40th Conference on Computer Communications (INFOCOM). pp 1–9

  • Wood JR, Wood LE (2008) Card sorting: current practices and beyond. J Usability Stud 4(1):1–6

    Google Scholar 

  • Wu M, Zhang Y, Liu J, Wang S, Zhang Z, Xia X, Mao X (2022) On the way to microservices:Exploring problems and solutions from online q &a community. In: Proceedings of the 29th International Conference on Software Analysis, Evolution and Reengineering (SANER). pp 432–443

  • Xu T, Jin L, Fan X, Zhou Y, Pasupathy S, Talwadker R (2015) Hey, you have given me too many knobs!: Understanding and dealing with over-designed configuration in system software. In: Proceedings of the 10th Joint Meeting on Foundations of Software Engineering (ESEC/FSE). pp 307–319

  • Zhang S, Ernst MD (2013) Automated diagnosis of software configuration errors. In: Proceedings of the 35th International Conference on Software Engineering (ICSE). pp 312–321

  • Zhang S, Ernst MD (2014) Which configuration option should i change? In: Proceedings of the 36th International Conference on Software Engineering (ICSE). pp 152–163

Download references

Acknowledgements

We would like to thank the service mesh expert who provided feedback on our findings. Their statements are accounts of personal experience and opinion, and are in no means relate to their current or past affiliations.

Author information

Authors and Affiliations

  1. School of Computing, Queen’s University, Kingston, Ontario, Canada

    Yihao Chen, Eduardo Fernandes & Ahmed E. Hassan

  2. The Maersk Mc-Kinney Moller Institute, University of Southern Denmark, Odense, Syddanmark, Denmark

    Bram Adams

Authors
  1. Yihao Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Eduardo Fernandes
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bram Adams
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ahmed E. Hassan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yihao Chen.

Ethics declarations

Conflicts of Interest

All authors declare that they have no conflicts of interest.

Additional information

Communicated by: Xin Peng.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix A: Index of Q &A Posts

Appendix A: Index of Q &A Posts

Table 9 Selected Sample Post URLs
Full size table

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, Y., Fernandes, E., Adams, B. et al. On practitioners’ concerns when adopting service mesh frameworks. Empir Software Eng 28, 113 (2023). https://doi.org/10.1007/s10664-023-10348-1

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10664-023-10348-1

Keywords

Search

Navigation