Efficient Migration of Large-Memory VMs Using Private Virtual Memory

  • Yuji Muraoka
  • Kenichi KouraiEmail author
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 1035)


Recently, Infrastructure-as-a-Service clouds provide virtual machines (VMs) with a large amount of memory. Such large-memory VMs can be migrated to other hosts on host maintenance, but it is costly to always preserve hosts with sufficient free memory as the destination of VM migration. Using virtual memory in destination hosts is a possible solution, but the performance of VM migration largely degrades because traditional general-purpose virtual memory causes frequent paging during the migration. This paper proposes VMemDirect, which achieves efficient migration of large-memory VMs using private virtual memory. VMemDirect creates private swap space for each VM on fast NVMe SSDs. Then it transfers likely accessed memory data to physical memory and the other data to the private swap space directly. This direct memory transfer can completely avoid paging during VM migration. We have implemented VMemDirect in KVM and showed that the performance of VM migration and the migrated VM was improved dramatically.



The research results have been achieved by the “Resilient Edge Cloud Designed Network (19304),” the Commissioned Research of National Institute of Information and Communications Technology (NICT), Japan.


  1. 1.
    Amit, N., Tsafrir, D., Schuster, A.: VSwapper: a memory swapper for virtualized environments. In: Proceedings of ACM International Conference on Architectural Support for Programming Languages and Operating Systems, pp. 349–366 (2014)Google Scholar
  2. 2.
    Banerjee, I., Moltmann, P., Tati, K., Venkatasubramanian, R.: VMware ESX memory resource management: swap. VMware Tech. J. 3(1), 48–56 (2014)Google Scholar
  3. 3.
    Deshpande, U., Chan, D., Guh, T., Edouard, J., Gopalan, K., Bila, N.: Agile live migration of virtual machines. In: Proceedings of IEEE International Parallel and Distributed Processing Symposium (2016)Google Scholar
  4. 4.
    Fitzpatrick, B.: memcached – a distributed memory object caching system.
  5. 5.
    Gu, J., Lee, Y., Zhang, Y., Chowdhury, M., Shin, K.: Efficient memory disaggregation with Infiniswap. In: Proceedings of USENIX Symposium Networked Systems Design and Implementation (2017)Google Scholar
  6. 6.
    Liang, S., Noronha, R., Panda, D.: Swapping to remote memory over InfiniBand: an approach using a high performance network block device. In: Proceedings of IEEE Cluster Computing (2005)Google Scholar
  7. 7.
    Saxena, M., Swift, M.: FlashVM: virtual memory management on flash. In: Proceedings of USENIX Annual Technical Conference (2010)Google Scholar
  8. 8.
    Suetake, M., Kashiwagi, T., Kizu, H., Kourai, K.: S-memV: split migration of large-memory virtual machines in IaaS clouds. In: Proceedings of IEEE International Conference Cloud Computing, pp. 285–293 (2018)Google Scholar
  9. 9.
    Suetake, M., Kizu, H., Kourai, K.: Split migration of large memory virtual machines. In: Proceedings of ACM SIGOPS Asia-Pacific Workshop of Systems (2016)Google Scholar
  10. 10.
    Suzuki, J., Baba, T., Hidaka, Y., Higuchi, J., Kami, N., Uchida, S., Takahashi, M., Sugawara, T., Yoshikawa, T.: Adaptive memory system over Ethernet. In: Proceedings of USENIX Workshop on Hot Topics in Storage and File Systems (2010)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Kyushu Institute of TechnologyIizukaJapan

Personalised recommendations