Abstract
Autoscaling enables container cluster orchestrators to automatically adjust computational resources, such as containers and Virtual Machines (VMs), to handle fluctuating workloads effectively. This adaptation can involve modifying the amount of resources (horizontal scaling) or adjusting their computational capacity (vertical scaling). The motivation for our work stems from the limitations of previous autoscaling approaches, which are either partial (scaling containers or VMs, but not both) or excessively complex to be used in real systems. This complexity arises from their use of models with a large number of variables and the addressing of two simultaneous challenges: achieving the optimal deployment for a single scheduling window and managing the transition between successive scheduling windows. We propose an Integer Linear Programming (ILP) model to address the challenge of autoscaling containers and VMs jointly, both horizontally and vertically, to minimize deployment costs. This model is designed to be used with predictive autoscalers and be solved in a reasonable time, even for large clusters. To this end, improvements and reasonable simplifications with respect to previous models have been carried out to drastically reduce the size of the resource allocation problem. Furthermore, the proposed model provides an enhanced representation of system performance in comparison to previous approaches. A tool called Conlloovia has been developed to implement this model. To evaluate its performance, we have conducted a comprehensive assessment, comparing it with two heuristic allocators with different problem sizes. Our findings indicate that Conlloovia consistently demonstrates lower deployment costs in a significant number of cases. Conlloovia has also been evaluated with a real application, using synthetic and real workload traces, as well as different scheduling windows, with deployment costs approximately 20% lower than heuristic allocators.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Availability of Data and Materials
The datasets generated and analysed during the current study are available on the GitHub repository https://github.com/asi-uniovi/conlloovia_exp.
References
Apache Software Foundation: Apache Mesos. https://mesos.apache.org/ (2023). Accessed 09 May 2023
Docker: Docker Swarm. https://docs.docker.com/engine/swarm/ (2023). Accessed 09 May 2023
Cloud Native Computing Foundation: Kubernetes. https://kubernetes.io/ (2023). Accessed 09 May 2023
Lorido-Botran, T., Miguel-Alonso, J., Lozano, J.A.: A Review of Auto-scaling Techniques for Elastic Applications in Cloud Environments. J. Grid Comput. 12(4), 559–592 (2014). https://doi.org/10.1007/s10723-014-9314-7
Guerrero, C., Lera, I., Juiz, C.: Genetic Algorithm for Multi-Objective Optimization of Container Allocation in Cloud Architecture. J. Grid Comput. 16(1), 113–135 (2018). https://doi.org/10.1007/s10723-017-9419-x
Fourati, M.H., Marzouk, S., Jmaiel, M.: A Review of Container level Autoscaling for Microservices-based Applications. In: 2021 IEEE 30th International Conference on Enabling Technologies: Infrastructure for Collaborative Enterprises (WETICE), pp. 17–22. IEEE, Bayonne, France (2021). https://doi.org/10.1109/WETICE53228.2021.00015
Zhong, Z., Buyya, R.: A Cost-Efficient Container Orchestration Strategy in Kubernetes-Based Cloud Computing Infrastructures with Heterogeneous Resources. ACM Trans. Internet Technol. 20(2), 1–24 (2020). https://doi.org/10.1145/3378447
Srirama, S.N., Adhikari, M., Paul, S.: Application deployment using containers with auto-scaling for microservices in cloud environment. J. Netw. Comput. Appl. 160, 102629 (2020). https://doi.org/10.1016/j.jnca.2020.102629
Rzadca, K., Findeisen, P., Swiderski, J., Zych, P., Broniek, P., Kusmierek, J., Nowak, P., Strack, B., Witusowski, P., Hand, S., Wilkes, J.: Autopilot: workload autoscaling at Google. In: Proceedings of the Fifteenth European Conference on Computer Systems, pp. 1–16. ACM, Heraklion Greece (2020). https://doi.org/10.1145/3342195.3387524
Kan, C.: DoCloud: An elastic cloud platform for Web applications based on Docker. In: 2016 18th International Conference on Advanced Communication Technology (ICACT), pp. 1–1. IEEE, PyeongChang, Korea (South) (2016). https://doi.org/10.1109/ICACT.2016.7423439
Kiss, T., DesLauriers, J., Gesmier, G., Terstyanszky, G., Pierantoni, G., Oun, O.A., Taylor, S.J.E., Anagnostou, A., Kovacs, J.: A cloud-agnostic queuing system to support the implementation of deadline-based application execution policies. Future Gener. Comput. Syst. 101, 99–111 (2019). https://doi.org/10.1016/j.future.2019.05.062
Kovács, J.: Supporting Programmable Autoscaling Rules for Containers and Virtual Machines on Clouds. J. Grid Comput. 17(4), 813–829 (2019). https://doi.org/10.1007/s10723-019-09488-w
Taherizadeh, S., Stankovski, V.: Dynamic Multi-level Auto-scaling Rules for Containerized Applications. Comput. J. 62(2), 174–197 (2019). https://doi.org/10.1093/comjnl/bxy043
Fourati, M.H., Marzouk, S., Jmaiel, M., Guerout, T.: Docker-C2A: Cost-Aware Autoscaler of Docker Containers for Microservices-based Applications. Adv. Sci. Technol. Eng. Syst. J. 5(6), 972–980 (2020). https://doi.org/10.25046/aj0506116
Fourati, M.H., Marzouk, S., Jmaiel, M.: EPMA: Elastic Platform for Microservices-based Applications: Towards Optimal Resource Elasticity. J. Grid Comput. 20(1), 6 (2022). https://doi.org/10.1007/s10723-021-09597-5
Aldwyan, Y.: Intelligent scaling of container-based web applications in geographically distributed clouds. PhD thesis, University of Melbourne (2021)
Wang, M., Zhang, D., Wu, B.: A Cluster Autoscaler Based on Multiple Node Types in Kubernetes. In: 2020 IEEE 4th Information Technology, Networking, Electronic and Automation Control Conference (ITNEC), pp. 575–579. IEEE, Chongqing, China (2020). https://doi.org/10.1109/ITNEC48623.2020.9084706
Imdoukh, M., Ahmad, I., Alfailakawi, M.G.: Machine learning-based auto-scaling for containerized applications. Neural. Comput. Appl. 32(13), 9745–9760 (2020). https://doi.org/10.1007/s00521-019-04507-z
Khaleq, A.A., Ra, I.: Intelligent Autoscaling of Microservices in the Cloud for Real-Time Applications. IEEE Access 9, 35464–35476 (2021). https://doi.org/10.1109/ACCESS.2021.3061890
Dogani, J., Khunjush, F., Seydali, M.: KAGRUED: A Container Autoscaling Technique for Cloud-based Web Applications in Kubernetes Using Attention-based GRU Encoder-Decoder. J. Grid Comput. 20(4), 40 (2022). https://doi.org/10.1007/s10723-022-09634-x
Toka, L., Dobreff, G., Fodor, B., Sonkoly, B.: Machine Learning-Based Scaling Management for Kubernetes Edge Clusters. IEEE Trans. Netw. Service Manag. 18(1), 958–972 (2021). https://doi.org/10.1109/TNSM.2021.3052837
Rattihalli, G., Govindaraju, M., Lu, H., Tiwari, D.: Exploring Potential for Non-Disruptive Vertical Auto Scaling and Resource Estimation in Kubernetes. In: 2019 IEEE 12th International Conference on Cloud Computing (CLOUD), pp. 33–40. IEEE, Milan, Italy (2019). https://doi.org/10.1109/CLOUD.2019.00018
Al-Dhuraibi, Y., Zalila, F., Djarallah, N., Merle, P.: Coordinating Vertical Elasticity of both Containers and Virtual Machines:. In: Proceedings of the 8th International Conference on Cloud Computing and Services Science, pp. 322–329. SCITEPRESS - Science and Technology Publications, Funchal, Madeira, Portugal (2018). https://doi.org/10.5220/0006652403220329
Yadav, M.P., Pal, N., Yadav, D.K.: Resource provisioning for containerized applications. Clust. Comput. 24(4), 2819–2840 (2021). https://doi.org/10.1007/s10586-021-03293-5
Rossi, F., Nardelli, M., Cardellini, V.: Horizontal and Vertical Scaling of Container–Based Applications Using Reinforcement Learning. In: 2019 IEEE 12th International Conference on Cloud Computing (CLOUD), pp. 329–338. IEEE, Milan, Italy (2019). https://doi.org/10.1109/CLOUD.2019.00061
Podolskiy, V., Jindal, A., Gerndt, M.: Multilayered Autoscaling Performance Evaluation: Can Virtual Machines and Containers Co-Scale? Int. J. Appl. Math. Comput. Sci. 29(2), 227–244 (2019). https://doi.org/10.2478/amcs-2019-0017
Ramirez, Y.M., Podolskiy, V., Gerndt, M.: Capacity-Driven Scaling Schedules Derivation for Coordinated Elasticity of Containers and Virtual Machines. In: 2019 IEEE International Conference on Autonomic Computing (ICAC), pp. 177–186. IEEE, Umea, Sweden (2019). https://doi.org/10.1109/ICAC.2019.00029
Hoenisch, P., Weber, I., Schulte, S., Zhu, L., Fekete, A.: Four-Fold Auto-Scaling on a Contemporary Deployment Platform Using Docker Containers. In: Barros, A., Grigori, D., Narendra, N.C., Dam, H.K. (eds.) Service-Oriented Computing vol. 9435, pp. 316–323. Springer, Berlin, Heidelberg (2015). https://doi.org/10.1007/978-3-662-48616-0_20
Nardelli, M., Cardellini, V., Casalicchio, E.: Multi-Level Elastic Deployment of Containerized Applications in Geo-Distributed Environments. In: 2018 IEEE 6th International Conference on Future Internet of Things and Cloud (FiCloud), pp. 1–8. IEEE, Barcelona, Spain (2018). https://doi.org/10.1109/FiCloud.2018.00009
Wais, A.: Optimizing Container Elasticity for Microservices in Hybrid Clouds. Master’s thesis, Institut für Information Systems Engineering, TU Wien (2021). https://doi.org/10.34726/HSS.2021.77600
Sheganaku, G., Schulte, S., Waibel, P., Weber, I.: Cost-efficient auto-scaling of container-based elastic processes. Future Gener. Comput. Syst. 138, 296–312 (2023). https://doi.org/10.1016/j.future.2022.09.001
Forrest, J., Ralphs, T., Santos, H.G., Vigerske, S., Hafer, L., Kristjansson, B., jpfasano, EdwinStraver, Lubin, M., rlougee, jpgoncal1, Jan-Willem, h-i-gassmann, Brito, S., Cristina, Saltzman, M., tosttost, Pitrus, B., Matsushima, F., to-st: coin-or/Cbc: Release releases/2.10.8. Zenodo (2022). https://doi.org/10.5281/zenodo.6522795
Jocher, G., Chaurasia, A., Stoken, A., Borovec, J., NanoCode012, Kwon, Y., Michael, K., TaoXie, Fang, J., imyhxy, Lorna, Yifu, Z., Wong, C., V, A., Montes, D., Wang, Z., Fati, C., Nadar, J., Laughing, UnglvKitDe, Sonck, V., tkianai, yxNONG, Skalski, P., Hogan, A., Nair, D., Strobel, M., Jain, M.: ultralytics/yolov5: v7.0 - YOLOv5 SOTA Realtime Instance Segmentation. Zenodo (2022). https://doi.org/10.5281/zenodo.7347926
Grafana Labs: k6. https://k6.io (2023). Accessed 18 Jul 2023
Fehling, C., Leymann, F., Retter, R.: Cloud Computing Patterns: Fundamentals to Design, Build, and Manage Cloud Applications. Springer, Wien, Austria (2014)
Weng, Q., Xiao, W., Yu, Y., Wang, W., Wang, C., He, J., Li, Y., Zhang, L., Lin, W., Ding, Y.: MLaaS in the wild: Workload analysis and scheduling in large-scale heterogeneous GPU clusters. In: 19th {USENIX} Symposium on Networked Systems Design and Implementation ({NSDI} 22) (2022)
Cloud Native Computing Foundation: Kubernetes deployments. https://kubernetes.io/docs/concepts/workloads/controllers/deployment/ (2023). Accessed 09 May 2023
Cloud Native Computing Foundation: Horizontal Pod Autoscaler. https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/ (2023). Accessed 09 May 2023
Cloud Native Computing Foundation: Cluster Autoscaler. https://github.com/kubernetes/autoscaler/tree/master/cluster-autoscaler (2023). Accessed 09 May 2023
Funding
Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature. This research was funded by the project PID2021-124383OB-I00 of the Spanish National Plan for Research, Development and Innovation from the Spanish Ministerio de Ciencia e Innovación.
Author information
Authors and Affiliations
Contributions
J.E. proposed the original idea of this research, providing a first version of the ILP model for the joint autoscaling of containers and VMs. Likewise, he wrote the Conlloovia tool and carried out the experimental evaluation with Conlloovia. M.G. reviewed the technical literature. J.G. conducted the performance measurement of the prototype in an EKS cluster, providing the required performance data for the experimental evaluation. J.M.L. improved the ILP model and developed the heuristic allocators. J.L.D. developed the application prototype for the study of Conlloovia on a real system. All the authors participated in the writing, review and editing of the manuscript.
Corresponding author
Ethics declarations
Competing Interests
The authors declare no competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Entrialgo, J., García, M., García, J. et al. Joint Autoscaling of Containers and Virtual Machines for Cost Optimization in Container Clusters. J Grid Computing 22, 17 (2024). https://doi.org/10.1007/s10723-023-09732-4
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10723-023-09732-4