The Journal of Supercomputing

, Volume 74, Issue 7, pp 2956–2983 | Cite as

Resource optimization of container orchestration: a case study in multi-cloud microservices-based applications

  • Carlos Guerrero
  • Isaac Lera
  • Carlos Juiz


An approach to optimize the deployment of microservices-based applications using containers in multi-cloud architectures is presented. The optimization objectives are three: cloud service cost, network latency among microservices, and time to start a new microservice when a provider becomes unavailable. The decision variables are: the scale level of the microservices; their allocation in the virtual machines; the provider and virtual machine type selection; and the number of virtual machines. The experiments compare the optimization results between a Greedy First-Fit and a Non-dominated Sorting Genetic Algorithm II (NSGA-II). NSGA-II with a two-point crossover operator and three mutation operators obtained an overall improvement of 300% in regard to the greedy algorithm.


Microservices Cloud computing Container orchestration Genetic algorithm Multi-objective optimization 



This research was supported by the Spanish Government (Agencia Estatal de Investigación) and the European Commission (Fondo Europeo de Desarrollo Regional) through Grant Number TIN2017-88547-P (AEI/FEDER, UE).


  1. 1.
    Adam O, Lee YC, Zomaya AY (2017) Stochastic resource provisioning for containerized multi-tier web services in clouds. IEEE Trans Parallel Distrib Syst 28(7):2060–2073. CrossRefGoogle Scholar
  2. 2.
    Adamuthe AC, Pandharpatte RM, Thampi GT (2013) Multiobjective virtual machine placement in cloud environment. In: 2013 International Conference on Cloud Ubiquitous Computing Emerging Technologies, pp 8–13 .
  3. 3.
    Amato A, Martino B, Venticinque S (2013) Multi-objective genetic algorithm for multi-cloud brokering. In: Euro-Par 2013: Parallel Processing Workshops, pp 55–64Google Scholar
  4. 4.
    Balalaie A, Heydarnoori A, Jamshidi P (2016) Microservices architecture enables devops: migration to a cloud-native architecture. IEEE Softw 33(3):42–52. CrossRefGoogle Scholar
  5. 5.
    Baldini I, Castro P, Chang K, Cheng P, Fink S, Ishakian V, Mitchell N, Muthusamy V, Rabbah R, Slominski A, Suter P (2017) Serverless computing: current trends and open problems. In: Chaudhary S, Somani G, Buyya R (eds) Research Advances in Cloud Computing, Springer, Singapore, pp 1–20.
  6. 6.
    Bermejo B, Filiposka S, Juiz C, Gomez B, Guerrero C (2017) Improving the energy efficiency in cloud computing data centres through resource allocation techniques. In: Chaudhary S, Somani G, Buyya R (eds) Research Advances in Cloud Computing, Springer, Singapore, pp 211–236.
  7. 7.
    Bernstein D (2014) Containers and cloud: from lxc to docker to kubernetes. IEEE Cloud Comput 1(3):81–84. CrossRefGoogle Scholar
  8. 8.
    Campbell M (2017) Distributed scheduler hell. USENIX Association, SingaporeGoogle Scholar
  9. 9.
    Casalicchio E (2017) Autonomic orchestration of containers: problem definition and research challenges. In: 10th EAI International Conference on Performance Evaluation Methodologies and Tools. ACM.
  10. 10.
    de Alfonso C, Calatrava A, Molt G (2017) Container-based virtual elastic clusters. J Syst Softw 127:1–11.
  11. 11.
    de Paula LC, Soares AS, de Lima TW, Filho AR, Coelho CJ (2016) Variable selection for multivariate calibration in chemometrics: a real-world application with building blocks disruption problem. In: Proceedings of the 2016 on Genetic and Evolutionary Computation Conference Companion, GECCO ’16 Companion. ACM, New York, NY, USA, pp 1031–1034.
  12. 12.
    Deb K, Pratap A, Agarwal S, Meyarivan T (2002) A fast and elitist multiobjective genetic algorithm: Nsga-II. Trans Evol Comput 6(2):182–197. CrossRefGoogle Scholar
  13. 13.
    Fard H, Prodan R, Fahringer T (2013) A truthful dynamic workflow scheduling mechanism for commercial multicloud environments. IEEE Trans Parallel Distrib Syst 24(6):1203–1212. CrossRefGoogle Scholar
  14. 14.
    Fazio M, Celesti A, Ranjan R, Liu C, Chen L, Villari M (2016) Open issues in scheduling microservices in the cloud. IEEE Cloud Comput 3(5):81–88. CrossRefGoogle Scholar
  15. 15.
    Frey S, Fittkau F, Hasselbring W (2013) Search-based genetic optimization for deployment and reconfiguration of software in the cloud. In: Proceedings of the 2013 International Conference on Software Engineering, ICSE ’13. IEEE Press, Piscataway, NJ, USA, pp 512–521.
  16. 16.
    Frincu ME, Craciun C (2011) Multi-objective meta-heuristics for scheduling applications with high availability requirements and cost constraints in multi-cloud environments. In: 2011 Fourth IEEE International Conference on Utility and Cloud Computing, pp 267–274Google Scholar
  17. 17.
    Fundation L (2018) Kubernetes—production-grade container orchestration.
  18. 18.
    Grefenstette JJ (1986) Optimization of control parameters for genetic algorithms. IEEE Trans Syst Man Cybern 16(1):122–128. CrossRefGoogle Scholar
  19. 19.
    Grozev N, Buyya R (2014) Inter-cloud architectures and application brokering: taxonomy and survey. Softw Pract Exp 44(3):369–390. CrossRefGoogle Scholar
  20. 20.
    Guan X, Wan X, Choi BY, Song S, Zhu J (2016) Application oriented dynamic resource allocation for data centers using docker containers. IEEE Commun Lett 21(99):1. Google Scholar
  21. 21.
    Guerrero C, Lera I, Juiz C (2018) Genetic algorithm for multi-objective optimization of container allocation in cloud architecture. J Grid Comput 16(1):113–135. CrossRefGoogle Scholar
  22. 22.
    Guerrero C, Lera I, Juiz C (2018) Migration-aware genetic optimization for mapreduce scheduling and replica placement in hadoop. J Grid Comput. Google Scholar
  23. 23.
    Guzek M, Bouvry P, Talbi EG (2015) A survey of evolutionary computation for resource management of processing in cloud computing. IEEE Comput Intell Mag 10(2):53–67. CrossRefGoogle Scholar
  24. 24.
    Harter T, Salmon B, Liu R, Arpaci-Dusseau AC, Arpaci-Dusseau RH (2016) Slacker: fast distribution with lazy docker containers. In: 14th USENIX Conference on File and Storage Technologies (FAST 16). USENIX Association, Santa Clara, CA, pp 181–195.
  25. 25.
    Hashicorp: Nomad by hashicorp (2018).
  26. 26.
    Heilig L, Lalla-Ruiz E, Voß S (2016) A cloud brokerage approach for solving the resource management problem in multi-cloud environments. Comput Ind Eng 95(C):16–26. CrossRefGoogle Scholar
  27. 27.
    Hoenisch P, Weber I, Schulte S, Zhu L, Fekete A (2015) Four-fold auto-scaling on a contemporary deployment platform using docker containers. In: Service-Oriented Computing: 13th International Conference. ICSOC 2015, Goa, India, November 16–19, 2015, Proceedings. Springer, Berlin, pp 316–323Google Scholar
  28. 28.
    Kaewkasi C, Chuenmuneewong K (2017) Improvement of container scheduling for docker using ant colony optimization. In: 2017 9th International Conference on Knowledge and Smart Technology (KST), pp 254–259.
  29. 29.
    Kang H, Le M, Tao S (2016) Container and microservice driven design for cloud infrastructure devops. In: 2016 IEEE International Conference on Cloud Engineering (IC2E), pp 202–211.
  30. 30.
    Kimovski D, Saurabh N, Stankovski V, Prodan R (2016) Multi-objective middleware for distributed VMI repositories in federated cloud environment. Scalable Comput Pract Exp 17(4):299–312.
  31. 31.
    Legillon F, Melab N, Renard D, Talbi EG (2013) Cost minimization of service deployment in a multi-cloud environment. In: 2013 IEEE Congress on Evolutionary Computation, pp 2580–2587.
  32. 32.
    Li J, Li D, Ye Y, Lu X (2015) Efficient multi-tenant virtual machine allocation in cloud data centers. Tsinghua Sci Technol 20(1):81–89. CrossRefGoogle Scholar
  33. 33.
    Li P, Nie H, Xu H, Dong L (2017) A minimum-aware container live migration algorithm in the cloud environment. Int J Bus Data Commun Netw 13(2):15–27. CrossRefGoogle Scholar
  34. 34.
    Lin CC, Liu P, Wu JJ (2011) Energy-aware virtual machine dynamic provision and scheduling for cloud computing. In: 2011 IEEE 4th International Conference on Cloud Computing, pp 736–737.
  35. 35.
    Lucas-Simarro JL, Moreno-Vozmediano R, Montero RS, Llorente IM (2013) Scheduling strategies for optimal service deployment across multiple clouds. Futur Gener Comput Syst 29(6):1431–1441 .
  36. 36.
    Mahmud R, Kotagiri R, Buyya R (2018) Fog computing: a taxonomy, survey and future directions. In: Di Martino B, Li KC, Yang L, Esposito A (eds) Internet of everything. Internet of things (technology, communications and computing), Springer, Singapore, pp 103–130.
  37. 37.
    Marler RT, Arora JS (2010) The weighted sum method for multi-objective optimization: new insights. Struct Multidiscip Optim 41(6):853–862. MathSciNetCrossRefzbMATHGoogle Scholar
  38. 38.
    Merkel D (2014) Docker: lightweight linux containers for consistent development and deployment. Linux J 2014(239).
  39. 39.
    Mitchell M (1998) An introduction to genetic algorithms. MIT Press, CambridgezbMATHGoogle Scholar
  40. 40.
    Nardelli M, Hochreiner C, Schulte S (2017) Elastic provisioning of virtual machines for container deployment. In: Proceedings of the 8th ACM/SPEC on International Conference on Performance Engineering Companion, ICPE ’17 Companion. ACM, New York, NY, USA, pp 5–10.
  41. 41.
    Newman S (2015) Building microservices: designing fine-grained systems. O’Reilly Media Inc., SebastopolGoogle Scholar
  42. 42.
    Panda SK, Jana PK (2015) Efficient task scheduling algorithms for heterogeneous multi-cloud environment. J Supercomput 71(4):1505–1533. CrossRefGoogle Scholar
  43. 43.
    Panda SK, Jana PK (2017) Sla-based task scheduling algorithms for heterogeneous multi-cloud environment. J Supercomput 73(6):2730–2762. CrossRefGoogle Scholar
  44. 44.
    Pascual JA, Lorido-Botrán T, Miguel-Alonso J, Lozano JA (2015) Towards a greener cloud infrastructure management using optimized placement policies. J Grid Comput 13(3):375–389. CrossRefGoogle Scholar
  45. 45.
    Peinl R, Holzschuher F, Pfitzer F (2016) Docker cluster management for the cloud—survey results and own solution. J Grid Comput 14(2):265–282. CrossRefGoogle Scholar
  46. 46.
    Pietrabissa A, Priscoli FD, Giorgio AD, Giuseppi A, Panfili M, Suraci V (2017) An approximate dynamic programming approach to resource management in multi-cloud scenarios. Int J Control 90(3):492–503. MathSciNetCrossRefzbMATHGoogle Scholar
  47. 47.
    Piraghaj SF, Dastjerdi AV, Calheiros RN, Buyya R (2015) A framework and algorithm for energy efficient container consolidation in cloud data centers. In: IEEE International Conference on Data Science and Data Intensive Systems, DSDIS 2015, Sydney, Australia, December 11–13, 2015, pp 368–375.
  48. 48.
    Skarlat O, Nardelli M, Schulte S, Borkowski M, Leitner P (2017) Optimized iot service placement in the fog. Serv Oriented Comput Appl 11(4):427–443. CrossRefGoogle Scholar
  49. 49.
    Sousa G, Rudametkin W, Duchien L (2016) Automated setup of multi-cloud environments for microservices applications. In: 2016 IEEE 9th International Conference on Cloud Computing (CLOUD), pp 327–334.
  50. 50.
    Tao Y, Wang X, Xu X, Chen Y (2017) Dynamic resource allocation algorithm for container-based service computing. In: 2017 IEEE 13th International Symposium on Autonomous Decentralized System (ISADS), pp 61–67.
  51. 51.
    Thnes J (2015) Microservices. IEEE Softw 32(1):116–116. CrossRefGoogle Scholar
  52. 52.
    Tordsson J, Montero RS, Moreno-Vozmediano R, Llorente IM (2012) Cloud brokering mechanisms for optimized placement of virtual machines across multiple providers. Future Gener Comput Syst 28(2):358–367.
  53. 53.
    Weaveworks Container Solutions (2018) Socks shop—a microservices demo application.
  54. 54.
    Wei G, Vasilakos AV, Zheng Y, Xiong N (2010) A game-theoretic method of fair resource allocation for cloud computing services. J Supercomput 54(2):252–269. CrossRefGoogle Scholar
  55. 55.
    Wolpert DH, Macready WG (1997) No free lunch theorems for optimization. Trans Evol Comput 1(1):67–82. CrossRefGoogle Scholar
  56. 56.
    Wu X, Gu Y, Tao J, Li G, Jayaraman PP, Sun D, Ranjan R, Zomaya A, Han J (2016) An online greedy allocation of vms with non-increasing reservations in clouds. J Supercomput 72(2):371–390. CrossRefGoogle Scholar
  57. 57.
    Yang X (2010) Firefly algorithm, stochastic test functions and design optimisation. Int J Bio Inspired Comput 2(2):78–84. CrossRefGoogle Scholar
  58. 58.
    Zhan ZH, Liu XF, Gong YJ, Zhang J, Chung HSH, Li Y (2015) Cloud computing resource scheduling and a survey of its evolutionary approaches. ACM Comput Surv 47(4):63:1–63:33. CrossRefGoogle Scholar
  59. 59.
    Ziafat H, Babamir SM (2017) A method for the optimum selection of datacenters in geographically distributed clouds. J Supercomput 73(9):4042–4081. CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Computer Science DepartmentBalearic Islands UniversityPalmaSpain

Personalised recommendations