Skip to main content
SpringerLink
Log in

Kubernetes-in-the-Loop: Enriching Microservice Simulation Through Authentic Container Orchestration

  • Conference paper
  • First Online:
Performance Evaluation Methodologies and Tools (VALUETOOLS 2023)

  • 145 Accesses

Abstract

Microservices deployed and managed by container orchestration frameworks like Kubernetes are the bases of modern cloud applications. In microservice performance modeling and prediction, simulations provide a lightweight alternative to experimental analysis, which requires dedicated infrastructure and a laborious setup. However, existing simulators cannot run realistic scenarios, as performance-critical orchestration mechanisms (like scheduling or autoscaling) are manually modeled and can consequently not be represented in their full complexity and configuration space. This work combines a state-of-the-art simulation for microservice performance with Kubernetes container orchestration. Hereby, we include the original implementation of Kubernetes artifacts enabling realistic scenarios and testing of orchestration policies with low overhead. In two experiments with Kubernetes’ kube-scheduler and cluster-autoscaler, we demonstrate that our framework can correctly handle different configurations of these orchestration mechanisms boosting both the simulation’s use cases and authenticity.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 49.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 64.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. https://github.com/DescartesResearch/misim-orchestration

  2. https://github.com/DescartesResearch/misim-k8s-adapter

  3. https://doi.org/10.24433/CO.4913288.v1

  4. https://github.com/joakimkistowski/HTTP-Load-Generator

  5. https://github.com/DescartesResearch/TeaStore

  6. https://research.spec.org/devopswg

  7. Aslanpour, M.S., Toosi, A.N., Taheri, J., Gaire, R.: Autoscalesim: A simulation toolkit for auto-scaling web applications in clouds. Simul. Modelling Pract. Theory (2021)

    Google Scholar 

  8. Bao, L., Wu, C., Bu, X., Ren, N., Shen, M.: Performance modeling and workflow scheduling of microservice-based applications in clouds. IEEE Trans. Parallel Distrib. Syst. (2019)

    Google Scholar 

  9. Becker, M., Becker, S., Meyer, J.: Simulizar: design-time modeling and performance analysis of self-adaptive systems. Softw. Eng. (2013)

    Google Scholar 

  10. Ben Ayed, M., Zouari, L., Abid, M.: Software in the loop simulation for robot manipulators. Eng., Technol. Appl. Sci. Res. (2017)

    Google Scholar 

  11. Bozóki, S., Szalontai, J., Pethő, D., Kocsis, I., Pataricza, A., Suskovics, P., Kovács, B.: Application of extreme value analysis for characterizing the execution time of resilience supporting mechanisms in Kubernetes. In: Dependable Computing—EDCC 2020 Workshops (2020)

    Google Scholar 

  12. Carrión, C.: Kubernetes Scheduling: Taxonomy, Ongoing Issues and Challenges. ACM Comput, Surv. (2022)

    Google Scholar 

  13. Cloud Native Computing Foundation: Cncf annual survey 2022 (2023). https://www.cncf.io/reports/cncf-annual-report-2022/

  14. Demers, S., Gopalakrishnan, P., Kant, L.: A generic solution to software-in-the-loop. In: MILCOM 2007—IEEE Military Communications Conference (2007)

    Google Scholar 

  15. Eismann, S., Walter, J., von Kistowski, J., Kounev, S.: Modeling of parametric dependencies for performance prediction of component-based software systems at run-time. In: IEEE International Conference on Software Architecture (ICSA) (2018)

    Google Scholar 

  16. Erb, B., Kargl, F.: Combining discrete event simulations and event sourcing. In: Proceedings of the 7th International ICST Conference on Simulation Tools and Techniques. SIMUTools 14 (2014)

    Google Scholar 

  17. Frank, S., Wagner, L., Hakamian, A., Straesser, M., van Hoorn, A.: Misim: a simulator for resilience assessment of microservice-based architectures. In: IEEE 22nd International Conference on Software Quality, Reliability and Security (QRS) (2022)

    Google Scholar 

  18. Friedman, N.: Kubernetes cluster autoscaler command line options (2023). https://gist.github.com/neerfri/4bd7477920cb33a2a229807ed10c29c2

  19. Hellerer, M., Schuster, M.J., Lichtenheldt, R.: Software-in-the-loop simulation of a planetary rover. In: The International Symposium on Artificial Intelligence, Robotics and Automation in Space (2016)

    Google Scholar 

  20. IBM Market Development & Insights: Microservices in the enterprise (2021). https://www.ibm.com/downloads/cas/OQG4AJAM

  21. Jawarneh, I.M.A., Bellavista, P., Bosi, F., Foschini, L., Martuscelli, G., Montanari, R., Palopoli, A.: Container orchestration engines: a thorough functional and performance comparison. In: IEEE International Conference on Communications (ICC) (2019)

    Google Scholar 

  22. Jindal, A., Podolskiy, V., Gerndt, M.: Performance modeling for cloud microservice applications. In: Proceedings of the ACM/SPEC International Conference on Performance Engineering (2019)

    Google Scholar 

  23. von Kistowski, J., Eismann, S., Schmitt, N., Bauer, A., Grohmann, J., Kounev, S.: Teastore: a micro-service reference application for benchmarking, modeling and resource management research. In: IEEE 26th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS) (2018)

    Google Scholar 

  24. Kubernetes Documentation: Assign pods to nodes using node affinity (2023). https://kubernetes.io/docs/tasks/configure-pod-container/assign-pods-nodes-using-node-affinity/

  25. Kubernetes Documentation: Kubernetes API concepts (2023). https://kubernetes.io/docs/reference/using-api/api-concepts/

  26. Kubernetes Documentation: The kubernetes cluster autoscaler (2023). https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/README.md

  27. Kubernetes Documentation: Kubernetes components (2023). https://kubernetes.io/docs/concepts/overview/components/

  28. Kubernetes Documentation: Scheduling framework (2023). https://kubernetes.io/docs/concepts/scheduling-eviction/scheduling-framework/

  29. Kubernetes SIG Cluster API: Kubernetes cluster API (2023). https://cluster-api.sigs.k8s.io/

  30. Kubernetes SIG Scheduling: Kube-scheduler-simulator (2023). https://github.com/kubernetes-sigs/kube-scheduler-simulator

  31. Law, A.M., Kelton, W.D.: Simulation modeling and analysis, vol. 3. Mcgraw-hill New York (2007)

    Google Scholar 

  32. Lechler, T., Page, B.: Desmo-j: An object oriented discrete simulation framework in java. In: Proceedings of the 11th European Simulation Symposium (ESS) (1999)

    Google Scholar 

  33. von Massow, R., van Hoorn, A., Hasselbring, W.: Performance simulation of runtime reconfigurable component-based software architectures. In: 5th European Conference on Software Architecture (ECSA) (2011)

    Google Scholar 

  34. Matarneh, R.J.: Self-adjustment time quantum in round robin algorithm depending on burst time of the now running processes. Am. J. Appl. Sci. (2009)

    Google Scholar 

  35. Mekki, M., Toumi, N., Ksentini, A.: Microservices configurations and the impact on the performance in cloud native environments. In: IEEE 47th Conference on Local Computer Networks (LCN) (2022)

    Google Scholar 

  36. Michelson, B.M.: Event-driven architecture overview. Patricia Seybold Group (2006)

    Google Scholar 

  37. Pan, Y., Chen, I., Brasileiro, F., Jayaputera, G., Sinnott, R.: A performance comparison of cloud-based container orchestration tools. In: IEEE International Conference on Big Knowledge (ICBK) (2019)

    Google Scholar 

  38. Schmoll, R., Fischer, T., Salah, H., Fitzek, F.H.P.: Comparing and evaluating application-specific boot times of virtualized instances. In: 2nd IEEE 5G World Forum (2019)

    Google Scholar 

  39. Straesser, M., Bauer, A., Leppich, R., Herbst, N., Chard, K., Foster, I., Kounev, S.: An empirical study of container image configurations and their impact on start times. In: 23rd IEEE International Symposium on Cluster, Cloud and Internet Computing (CCGrid) (2023)

    Google Scholar 

  40. Straesser, M., Grohmann, J., von Kistowski, J., Eismann, S., Bauer, A., Kounev, S.: Why is it not solved yet? challenges for production-ready autoscaling. In: Proceedings of the ACM/SPEC on International Conference on Performance Engineering (2022)

    Google Scholar 

  41. Straesser, M., Mathiasch, J., Bauer, A., Kounev, S.: A systematic approach for benchmarking of container orchestration frameworks. In: Proceedings of the ACM/SPEC International Conference on Performance Engineering (2023)

    Google Scholar 

  42. Tamiru, M.A., Tordsson, J., Elmroth, E., Pierre, G.: An experimental evaluation of the Kubernetes cluster autoscaler in the cloud. In: IEEE International Conference on Cloud Computing Technology and Science (CloudCom) (2020)

    Google Scholar 

  43. Tesfatsion, S.K., Klein, C., Tordsson, J.: Virtualization techniques compared: Performance, resource, and power usage overheads in clouds. In: ACM/SPEC International Conference on Performance Engineering (2018)

    Google Scholar 

  44. Truyen, E., Bruzek, M., Van Landuyt, D., Lagaisse, B., Joosen, W.: Evaluation of container orchestration systems for deploying and managing nosql database clusters. In: IEEE 11th International Conference on Cloud Computing (CLOUD) (2018)

    Google Scholar 

  45. Truyen, E., Van Landuyt, D., Lagaisse, B., Joosen, W.: Performance overhead of container orchestration frameworks for management of multi-tenant database deployments. In: Proceedings of the 34th ACM/SIGAPP Symposium on Applied Computing. SAC 19 (2019)

    Google Scholar 

  46. Valera, H.H.A., Dalmau, M., Roose, P., Larracoechea, J., Herzog, C.: Draceo: a smart simulator to deploy energy saving methods in microservices based networks. In: IEEE 29th International Conference on Enabling Technologies: Infrastructure for Collaborative Enterprises (WETICE) (2020)

    Google Scholar 

  47. VMware: K-bench (2023). https://github.com/vmware-tanzu/k-bench

  48. Wang, M., Zhang, D., Wu, B.: A cluster autoscaler based on multiple node types in Kubernetes. In: IEEE 4th Information Technology, Networking, Electronic and Automation Control Conference (ITNEC) (2020)

    Google Scholar 

  49. Xavier, B., Ferreto, T., Jersak, L.: Time provisioning evaluation of KVM, Docker and Unikernels in a cloud platform. In: 16th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (2016)

    Google Scholar 

  50. Zhang, Y., Gan, Y., Delimitrou, C.: \(\upmu \)qsim: enabling accurate and scalable simulation for interactive microservices. In: IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS) (2019)

    Google Scholar 

Download references

Acknowledgements

This work was funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) - 510552229. We thank the German Federal Ministry of Education and Research (dqualizer FKZ: 01IS22007B and Software Campus 2.0 — Microproject: DiSpel, FKZ: 01IS17051), the Google Cloud Research Credits program (GCP213506809) and our student researchers Lion Wagner and Lukas Mönch for supporting this work. The work was conducted in the context of the SPEC RG DevOps Performance Working Group [6].

Author information

Authors and Affiliations

  1. University of Würzburg, Würzburg, Germany

    Martin Straesser, Patrick Haas & Samuel Kounev

  2. University of Hamburg, Hamburg, Germany

    Sebastian Frank & André van Hoorn

  3. University of Stuttgart, Stuttgart, Germany

    Alireza Hakamian

Authors
  1. Martin Straesser
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Patrick Haas
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sebastian Frank
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Alireza Hakamian
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. André van Hoorn
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Samuel Kounev
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Martin Straesser .

Editor information

Editors and Affiliations

  1. University of Cambridge, Cambridge, UK

    Evangelia Kalyvianaki

  2. University of Southern California, Los Angeles, CA, USA

    Marco Paolieri

Rights and permissions

Reprints and permissions

Copyright information

© 2024 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Straesser, M., Haas, P., Frank, S., Hakamian, A., van Hoorn, A., Kounev, S. (2024). Kubernetes-in-the-Loop: Enriching Microservice Simulation Through Authentic Container Orchestration. In: Kalyvianaki, E., Paolieri, M. (eds) Performance Evaluation Methodologies and Tools. VALUETOOLS 2023. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 539. Springer, Cham. https://doi.org/10.1007/978-3-031-48885-6_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-48885-6_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-48884-9

  • Online ISBN: 978-3-031-48885-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation