Journal of Network and Systems Management

, Volume 26, Issue 4, pp 1034–1057 | Cite as

Flow-Aware Workload Migration in Data Centers

  • Yoann DesmouceauxEmail author
  • Sonia Toubaline
  • Thomas Clausen


In data centers, subject to workloads with heterogeneous (and sometimes short) lifetimes, workload migration is a way of attaining a more efficient utilization of the underlying physical machines. To not introduce performance degradation, such workload migration must take into account not only machine resources, and per-task resource requirements, but also application dependencies in terms of network communication. This paper presents a workload migration model capturing all of these constraints. A linear programming framework is developed allowing accurate representation of per-task resources requirements and inter-task network demands. Using this, a multi-objective problem is formulated to compute a re-allocation of tasks that (1) maximizes the total inter-task throughput, while (2) minimizing the cost incurred by migration and (3) allocating the maximum number of new tasks. A baseline algorithm, solving this multi-objective problem using the \(\varepsilon\)-constraint method is proposed, in order to generate the set of Pareto-optimal solutions. As this algorithm is compute-intensive for large topologies, a heuristic, which computes an approximation of the Pareto front, is then developed, and evaluated on different topologies and with different machine load factors. These evaluations show that the heuristic can provide close-to-optimal solutions, while reducing the solving time by one to two order of magnitudes.


Data center networking VM migration Application-aware allocation MILP Multi-objective optimization Pareto optimality 


  1. 1.
    Clark, C., Fraser, K., Hand, S., Hansen, J.G., Jul, E., Limpach, C., Pratt, I., Warfield, A.: Live migration of virtual machines. In: Proceedings of the 2nd Conference on Symposium on Networked Systems Design and Implementation, vol. 2, pp. 273–286. USENIX Association (2005)Google Scholar
  2. 2.
    Bolla, R., Chiappero, M., Rapuzzi, R., Repetto, M.: Seamless and transparent migration for tcp sessions. In: 2014 IEEE 25th Annual International Symposium on Personal, Indoor, and Mobile Radio Communication (PIMRC), pp. 1469–1473. IEEE (2014)Google Scholar
  3. 3.
    Nadgowda, S., Suneja, S., Bila, N., Isci, C.: Voyager: complete container state migration. In: 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS), pp. 2137–2142. IEEE (2017)Google Scholar
  4. 4.
    Cheng, D., Jiang, C., Zhou, X.: Heterogeneity-aware workload placement and migration in distributed sustainable datacenters. In: 2014 IEEE 28th International on Parallel and Distributed Processing Symposium, pp. 307–316. IEEE (2014)Google Scholar
  5. 5.
    Zeng, D., Gu, L., Guo, S.: Cost minimization for big data processing in geo-distributed data centers. In: Zeng, D., Gu, L., Guo, S. (eds.) Cloud Networking for Big Data, pp. 59–78. Springer, Cham (2015)CrossRefGoogle Scholar
  6. 6.
    Shrivastava, V., Zerfos, P., Lee, K.-W., Jamjoom, H., Liu, Y.-H., Banerjee, S.: Application-aware virtual machine migration in data centers. In: INFOCOM, 2011 Proceedings IEEE, pp. 66–70. IEEE (2011)Google Scholar
  7. 7.
    Huang, D., Gao, Y., Song, F., Yang, D., Zhang, H.: Multi-objective virtual machine migration in virtualized data center environments. In: 2013 IEEE International Conference on Communications (ICC), pp. 3699–3704. IEEE (2013)Google Scholar
  8. 8.
    Benson, T., Akella, A., Maltz, D.A.: Network traffic characteristics of data centers in the wild. In: Proceedings of the 10th ACM SIGCOMM Conference on Internet Measurement, pp. 267–280. ACM (2010)Google Scholar
  9. 9.
    Kandula, S., Sengupta, S., Greenberg, A., Patel, P., Chaiken, R.: The nature of data center traffic: measurements and analysis. In: Proceedings of the 9th ACM SIGCOMM Conference on Internet Measurement Conference, pp. 202–208. ACM (2009)Google Scholar
  10. 10.
    Meng, X., Pappas, V., Zhang, L.: Improving the scalability of data center networks with traffic-aware virtual machine placement. In: INFOCOM, 2010 Proceedings IEEE, pp. 1–9. IEEE (2010)Google Scholar
  11. 11.
    LaCurts, K., Deng, S., Goyal, A., Balakrishnan, H.: Choreo: network-aware task placement for cloud applications. In: Proceedings of the 2013 Conference on Internet Measurement Conference, pp. 191–204. ACM (2013)Google Scholar
  12. 12.
    Al-Fares, M., Radhakrishnan, S., Raghavan, B., Huang, N., Vahdat, A.: Hedera: dynamic flow scheduling for data center networks. NSDI 10, 19–19 (2010)Google Scholar
  13. 13.
    Ferreto, T.C., Netto, M.A.S., Calheiros, R.N., De Rose, C.A.F.: Server consolidation with migration control for virtualized data centers. Future Gener. Comput. Syst. 27, 1027–1034 (2011)CrossRefGoogle Scholar
  14. 14.
    Ghribi, C., Hadji, M., Zeghlache, D.: Energy efficient VM scheduling for cloud data centers: exact allocation and migration algorithms. In: 2013 13th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing. IEEE (2013)Google Scholar
  15. 15.
    Jin, H., Cheocherngngarn, T., Levy, D., Smith, A., Pan, D., Liu, J., Pissinou, N.: Joint host-network optimization for energy-efficient data center networking. In: 2013 IEEE 27th International Symposium on Parallel and Distributed Processing (IPDPS), pp. 623–634. IEEE (2013)Google Scholar
  16. 16.
    Liu, N., Dong, Z., Rojas-Cessa, R.: Task and server assignment for reduction of energy consumption in datacenters. In: 2012 11th IEEE International Symposium on Network Computing and Applications (NCA), pp. 171–174. IEEE (2012)Google Scholar
  17. 17.
    kakadia, D., Kopri, N., Varma, V.: Network-aware virtual machine consolidation for large data centers. In: NDM ’13 Proceedings of the Third International Workshop on Network-Aware Data Management. ACM (6) (2013)Google Scholar
  18. 18.
    Ahmad, R.W., Gani, A., Hamide, S.H.A., Shiraz, M., Yousafzai, A., Xia, F.: A survey on virtual machine migration and server consolidation frameworks for cloud data centers. J. Netw. Comput. Appl. 52, 11–25 (2015)CrossRefGoogle Scholar
  19. 19.
    Pires, F.L., Báran, B.: Virtual machine placement literature review. arXiv:1506.01509v1. Cited 4 June 2015
  20. 20.
    Ferdaus, M.H., Murshed, M., Calheiros, R.N., Buyya, R.: Network-aware virtual machine placement and migration in cloud data centers. In: Bagchi, S. (ed.) Emerging Research in Cloud Distributed Computer Systems, pp. 42–91. IGI Global, Hershey (2015)CrossRefGoogle Scholar
  21. 21.
    Usmani, Z., Singh, S.: A survey of virtual machine placement techniques in cloud data center. Procedia Comput. Sci. 78, 491–498 (2016)CrossRefGoogle Scholar
  22. 22.
    Fang, W., Liang, X., Li, S., Chiaraviglio, L., Xiong, N.: VMPlanner: optimizing virtual machine placement and traffic flow routing to reduce network power costs in cloud data centers. Comput. Netw. 57(1), 179–196 (2013)CrossRefGoogle Scholar
  23. 23.
    Chen, T., Gao, X., Chen, G.: Optized virtual machine placement with traffic-aware balancing in data ceter networks. Sci. Program. 6, 10 (2016)Google Scholar
  24. 24.
    Fang, S., Kanagavelu, R., Lee, B.-S., Foh, C.H., Aung, K.M.M.: Power-efficient virtual machine placement and migration in data centers. In: 2013 IEEE International Conference on Green Computing and Communications and IEEE Internet of Things and IEEE Cyber, Physical and Social Computing, pp. 1408–1413. IEEE Computer Society (2013)Google Scholar
  25. 25.
    Xu, J., Fortes, J.A.B.: Multi-objective virtual machine placement in virtualized data center environments. In: 2010 IEEE/ACM International Conference on Green Computing and Communications and 2010 IEEE/ACM International Conference on Cyber, Physical and Social Computing, pp. 179–188. IEEE Computer Society (2010)Google Scholar
  26. 26.
    Xu, J., Fortes, J.A.B.: A multi-objective approach to virtual machine management in datacenters. In: Proceeding ICAC’11 Proceedings of the 8th ACM International Conference on Autonomic Computing, pp. 225–234. ACM, New York (2011)Google Scholar
  27. 27.
    Gao, Y., Guan, H., Qi, Z., Hou, Y., Liu, L.: A multi-objective ant colony system algorithm for virtual machine placement in cloud computing. J. Comput. Syst. Sci. 79, 1230–1242 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  28. 28.
    Pires, F.L., Báran, B.: Multi-objective virtual machine placement with service level agreement. In: 2013 IEEE/ACM 6th International Conference on Utility and Cloud Computing, pp. 203–210. IEEE Computer Society (2013)Google Scholar
  29. 29.
    Pires, F.L., Báran, B.: Virtual machine placement. A multi-objective approach. In: Latin American Symposium of Infrastructure, Hadward, and Software, pp. 77–84. IEEE (2013)Google Scholar
  30. 30.
    Ehrgott, M.: Multicriteria Optimization. Springer, New York (2006)zbMATHGoogle Scholar
  31. 31.
    Ahuja, R.K., Magnanti, T.L., Orlin, J.B., Reddy, M.R.: Applications of network optimization. Handb. Oper. Res. Manag. Sci. 7, 1–83 (1995)MathSciNetzbMATHGoogle Scholar
  32. 32.
    Laumanns, M., Thiele, L., Zitzler, E.: An adaptive scheme to generate the pareto front based on the epsilon-constraint method. In: Dagstuhl Seminar Proceedings. Schloss Dagstuhl-Leibniz-Zentrum für Informatik (2005)Google Scholar
  33. 33.
    Gurobi Optimization Inc.: Gurobi optimizer reference manual. (2017). Accessed 23 Jul 2017

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.École PolytechniquePalaiseauFrance
  2. 2.Cisco Systems Paris Innovation and Research Laboratory (PIRL)Issy-les-MoulineauxFrance
  3. 3.Université Paris-DauphinePSL Research UniversityParisFrance

Personalised recommendations