ECSched: Efficient Container Scheduling on Heterogeneous Clusters

  • Yang HuEmail author
  • Huan Zhou
  • Cees de Laat
  • Zhiming Zhao
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11014)


Operating system (OS) containers are becoming increasingly popular in cloud computing for improving productivity and code portability. However, container scheduling on large heterogeneous cluster is quite challenging. Recent research on cluster scheduling focuses either on scheduling speed to quickly assign resources, or on scheduling quality to improve application performance and cluster utilization. In this paper, we propose ECSched, an efficient container scheduler that can make high-quality and fast placement decisions for concurrent deployment requests on heterogeneous clusters. We map the scheduling problem to a graphic data structure and model it as minimum cost flow problem (MCFP). We implement ECSched based on our cost model, which encodes the deployment requirements of requested containers. In the evaluation, we show that ECSched exceeds the placement quality of existing container schedulers with relatively small overheads, while providing \(1.1{\times }\) better resource efficiency and \(1.3{\times }\) lower average container completion time.



This research has received funding from the European Union’s Horizon 2020 research and innovation program under grant agreements 643963 (SWITCH project), 654182 (ENVRIPLUS project) and 676247 (VRE4EIC project). The research is also funded by Chinese Scholarship Council.


  1. 1.
    Amazon web services.
  2. 2.
  3. 3.
    Google kubernetes.
  4. 4.
  5. 5.
    Ahuja, R.K., Magnanti, T.L., Orlin, J.B.: Network Flows. Elsevier, New York (2014)zbMATHGoogle Scholar
  6. 6.
    Ajiro, Y., Tanaka, A.: Improving packing algorithms for server consolidation. In: International CMG Conference, vol. 253 (2007)Google Scholar
  7. 7.
    Baldin, I., et al.: ExoGENI: a multi-domain infrastructure-as-a-service testbed. In: McGeer, R., Berman, M., Elliott, C., Ricci, R. (eds.) The GENI Book, pp. 279–315. Springer, Cham (2016). Scholar
  8. 8.
    Burns, B., Grant, B., Oppenheimer, D., Brewer, E., Wilkes, J.: Borg, omega, and kubernetes. Commun. ACM 59(5), 50–57 (2016)CrossRefGoogle Scholar
  9. 9.
    Delimitrou, C., Sanchez, D., Kozyrakis, C.: Tarcil: reconciling scheduling speed and quality in large shared clusters. In: Proceedings of the Sixth ACM Symposium on Cloud Computing, pp. 97–110. ACM (2015)Google Scholar
  10. 10.
    Gog, I., Schwarzkopf, M., Gleave, A., Watson, R.N., Hand, S.: Firmament: fast, centralized cluster scheduling at scale. USENIX (2016)Google Scholar
  11. 11.
    Goldberg, A.V., Tarjan, R.E.: Finding minimum-cost circulations by canceling negative cycles. J. ACM (JACM) 36(4), 873–886 (1989)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Hindman, B., et al.: Mesos: a platform for fine-grained resource sharing in the data center. In: NSDI, vol. 11, p. 22 (2011)Google Scholar
  13. 13.
    Hu, Y., et al.: Deadline-aware deployment for time critical applications in clouds. In: Rivera, F.F., Pena, T.F., Cabaleiro, J.C. (eds.) Euro-Par 2017. LNCS, vol. 10417, pp. 345–357. Springer, Cham (2017). Scholar
  14. 14.
    Isard, M., Prabhakaran, V., Currey, J., Wieder, U., Talwar, K., Goldberg, A.: Quincy: fair scheduling for distributed computing clusters. In: Proceedings of the ACM SIGOPS 22nd Symposium on Operating Systems Principles, pp. 261–276. ACM (2009)Google Scholar
  15. 15.
    Lee, S., et al.: Validating heuristics for virtual machines consolidation. Microsoft Research, MSR-TR-2011-9 pp. 1–14 (2011)Google Scholar
  16. 16.
    Lodi, A., Martello, S., Vigo, D.: Recent advances on two-dimensional bin packing problems. Discret. Appl. Math. 123(1), 379–396 (2002)MathSciNetCrossRefGoogle Scholar
  17. 17.
    Mi, H., Wang, H., Yin, G., Zhou, Y., Shi, D., Yuan, L.: Online self-reconfiguration with performance guarantee for energy-efficient large-scale cloud computing data centers. In: 2010 IEEE International Conference on Services Computing (SCC), pp. 514–521. IEEE (2010)Google Scholar
  18. 18.
    Orlin, J.B.: A faster strongly polynomial minimum cost flow algorithm. Oper. Res. 41(2), 338–350 (1993)MathSciNetCrossRefGoogle Scholar
  19. 19.
    Ousterhout, K., Wendell, P., Zaharia, M., Stoica, I.: Sparrow: distributed, low latency scheduling. In: Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles, pp. 69–84. ACM (2013)Google Scholar
  20. 20.
    Panigrahy, R., Talwar, K., Uyeda, L., Wieder, U.: Heuristics for vector bin packing (2011).
  21. 21.
    Reiss, C., Tumanov, A., Ganger, G.R., Katz, R.H., Kozuch, M.A.: Heterogeneity and dynamicity of clouds at scale: Google trace analysis. In: Proceedings of the Third ACM Symposium on Cloud Computing, p. 7. ACM (2012)Google Scholar
  22. 22.
    Stillwell, M., Schanzenbach, D., Vivien, F., Casanova, H.: Resource allocation algorithms for virtualized service hosting platforms. J. Parallel Distrib. Comput. 70(9), 962–974 (2010)CrossRefGoogle Scholar
  23. 23.
    Taherizadeh, S., Jones, A.C., Taylor, I., Zhao, Z., Stankovski, V.: Monitoring self-adaptive applications within edge computing frameworks: a state-of-the-art review. J. Syst. Softw. 136, 19–38 (2018)CrossRefGoogle Scholar
  24. 24.
    Wang, J., et al.: Planning virtual infrastructures for time critical applications with multiple deadline constraints. Future Gen. Comput. Syst. 75, 365–375 (2017)CrossRefGoogle Scholar
  25. 25.
    Xu, J., Fortes, J.A.: Multi-objective virtual machine placement in virtualized data center environments. In: Proceedings of the 2010 IEEE/ACM International Conference on Green Computing and Communications & International Conference on Cyber, Physical and Social Computing, pp. 179–188. IEEE Computer Society (2010)Google Scholar
  26. 26.
    Zhan, Z.H., Liu, X.F., Gong, Y.J., Zhang, J., Chung, H.S.H., Li, Y.: Cloud computing resource scheduling and a survey of its evolutionary approaches. ACM Comput. Surv. (CSUR) 47(4), 63 (2015)CrossRefGoogle Scholar
  27. 27.
    Zhao, Z., et al.: A software workbench for interactive, time critical and highly self-adaptive cloud applications (switch). In: 2015 15th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid), pp. 1181–1184. IEEE (2015)Google Scholar
  28. 28.
    Zhou, H., et al.: Fast resource co-provisioning for time critical applications based on networked infrastructures. In: 2016 IEEE 9th International Conference on Cloud Computing (CLOUD), pp. 802–805. IEEE (2016)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Yang Hu
    • 1
    • 2
    Email author
  • Huan Zhou
    • 1
  • Cees de Laat
    • 1
  • Zhiming Zhao
    • 1
  1. 1.University of AmsterdamAmsterdamThe Netherlands
  2. 2.National University of Defense TechnologyChangshaChina

Personalised recommendations