Journal of Grid Computing

, Volume 10, Issue 1, pp 69–84 | Cite as

VMMB: Virtual Machine Memory Balancing for Unmodified Operating Systems

  • Changwoo Min
  • Inhyeok Kim
  • Taehyoung Kim
  • Young Ik Eom


Virtualization technology has been widely adopted in Internet hosting centers and cloud-based computing services, since it reduces the total cost of ownership by sharing hardware resources among virtual machines (VMs). In a virtualized system, a virtual machine monitor (VMM) is responsible for allocating physical resources such as CPU and memory to individual VMs. Whereas CPU and I/O devices can be shared among VMs in a time sharing manner, main memory is not amendable to such multiplexing. Moreover, it is often the primary bottleneck in achieving higher degrees of consolidation. In this paper, we present VMMB (Virtual Machine Memory Balancer), a novel mechanism to dynamically monitor the memory demand and periodically re-balance the memory among the VMs. VMMB accurately measures the memory demand with low overhead and effectively allocates memory based on the memory demand and the QoS requirement of each VM. It is applicable even to guest OS whose source code is not available, since VMMB does not require modifying guest kernel. We implemented our mechanism on Linux and experimented on synthetic and realistic workloads. Our experiments show that VMMB can improve performance of VMs that suffers from insufficient memory allocation by up to 3.6 times with low performance overhead (below 1%) for monitoring memory demand.


Virtualization Memory balancing LRU histogram Double paging 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Simons, J., Buell, J.: Virtualizing high performance computing. SIGOPS Oper. Syst. Rev. 44(4), 136–145 (2010)CrossRefGoogle Scholar
  2. 2.
    Lange, J., Pedretti, K., Dinda, P., Bae, C., Bridges, P., Soltero, P., Merritt, A.: Minimal-overhead virtualization of a large scale supercomputer. In: Proceedings of the 7th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments, pp. 169–180 (2011)Google Scholar
  3. 3.
    Iosup, A., Ostermann, S., Yigitbasi, M.N., Prodan, R., Fahringer, T., Epema, D.H.J.: Performance analysis of cloud computing services for many-tasks scientific computing. IEEE T Parall Distr 22(6), 931–945 (2010)CrossRefGoogle Scholar
  4. 4.
    Magenheimer, D.: Memory Overcommit Without the Commitment. Extended Abstract at the Xen Summit Boston 2008 (2008)Google Scholar
  5. 5.
    Zhao, W., Wang, Z., Luo, Y.: Dynamic memory balancing for virtual machines. SIGOPS Oper. Syst. Rev. 43(3), 37–47 (2009)CrossRefGoogle Scholar
  6. 6.
    Gupta, D., Lee, S., Vrable, M., Savage, S., Snoeren, A.C., Varghese, G., Voelker, G.M., Vahdat, A.: Difference engine: harnessing memory redundancy in virtual machines. Commun. ACM 53(10), 85–93 (2008)CrossRefGoogle Scholar
  7. 7.
    Barroso, L.A., Holzle, U.: The Datacenter as a Computer: An Introduction to the Design of Warehouse-Scale Machines. Morgan and Claypool Publishers (2009)Google Scholar
  8. 8.
    Waldspurger, C.A.: Memory resource management in VMware ESX server. SIGOPS Oper. Syst. Rev. 36(SI), 181–194 (2002)CrossRefGoogle Scholar
  9. 9.
    Scwidefsky, M., Franke, H., Mansell, R., Raj, H., Osisek, D., Choi, J.: Collaborative memory management in hosted linux environments. In: Proceedings of the Linux Symposium, vol. 2. Ottawa, Canada (2006)Google Scholar
  10. 10.
    Jones, S.T., Arpaci-Dusseau, A.C., Arpaci-Dusseau, R.H.: Geiger: monitoring the buffer cache in a virtual machine environment. SIGOPS Oper. Syst. Rev. 40(5), 14–24 (2006)CrossRefGoogle Scholar
  11. 11.
    Lu, P., Shen, K.: Virtual machine memory access tracing with hypervisor exclusive cache. In: Proceeding of the 2007 USENIX Annual Technical Conference, pp. 1–15. Berkeley, CA (2007)Google Scholar
  12. 12.
    Clark, C., Fraser, K., Hand, S., Hansen, J.G., Jul, E., Limpach, C., Pratt, I., Warfield, A.: Live migration of virtual machines, In: Proceedings of the 2nd Symposium on Networked Systems Design & Implementation, pp. 273–286 (2005)Google Scholar
  13. 13.
    Nelson, M., Lim, B.-H., Hutchins, G.: Fast transparent migration for virtual machines. In: Proceedings of USENIX Annual Technical Conference (2005)Google Scholar
  14. 14.
    Hines, M.R., Gopalan, K.: Post-copy based live virtual machine migration using adaptive pre-paging and dynamic self-ballooning. In: Proceedings of the International Conference on Virtual Execution Environments, pp. 51–60 (2009)Google Scholar
  15. 15.
    Hermenier, F., Lorca, X., Menaud, J.-M., Muller, G., Lawall, J.: Entropy: a ConsolidationManager for Clusters. In: Proceedings of the International Conference on Virtual Execution Environments, pp. 41–50 (2009)Google Scholar
  16. 16.
    Wood, T., Shenoy, P., Venkataramani, A.: Black-box and gray-box strategies for virtual machine migration, In: Proceedings of the 4th Symposium on Networked Systems Design & Implementation, pp. 229–242 (2007)Google Scholar
  17. 17.
    Stage, A., Setzer, T.: Network-aware migration control and scheduling of differentiated virtual machine workloads. In: Proceedings of ICSE Workshop on Software Engineering Challenges of Cloud Computing, pp. 9–14 (2009)Google Scholar
  18. 18.
    Andreolini, M., Casolari, S., Colajanni, M., Messori, M.: Dynamic Load Management of Virtual Machines in Cloud Architectures. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering 34(6), 201–214 (2010)CrossRefGoogle Scholar
  19. 19.
    Williams, D., Weatherspoon, H., Jamjoom, H., Liu, Y.: Overdriver: handling memory overload in an oversubscribed cloud. In: Proceedings of the 7th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments, pp. 205–216 (2011)Google Scholar
  20. 20.
    Mattson, R.L., Gecsei, J., Slutz, D., Traiger, I.L.: Evaluation techniques for storage hierarchies. IBM Syst. J. 9(2), 456–789 (1970)CrossRefGoogle Scholar
  21. 21.
    Goldberg, R.P., Hassinger, R.: The double paging anomaly. In: Proceedings of the National Computer Conference and Exposition, pp. 195–199 (1974)Google Scholar
  22. 22.
    Sherman, S.W., Brice, R.S.: Performance of a database manager in a virtual memory system. ACM Trans. Database Syst. 1(4), 317–343 (1976)CrossRefGoogle Scholar
  23. 23.
    Intel 64 and IA-32 Architectures Software Developers Manual vol. 3B: System Programming Guide, Part 2. Copyright 1997–2009 Intel Corporation.Google Scholar
  24. 24.
    Cormen, T.H., Leiserson, C.E., Rivest, R.L., Stein, C.: Introduction to Algorithms, 3rd edn, pp. 308–329. The MIT Press (2009)Google Scholar
  25. 25.
    Qureshi, M.K., Patt, Y.N.: Utility-based Cache Partitioning: A Low-overhead, High-performance, Runtime Mechanism to Partition Shared Caches. In: Proceedings of the 39th Annual IEEE/ACM International Symposium on Microarchitecture, pp. 423–432 (2006)Google Scholar
  26. 26.
    Bovet, D.P., Cesati, M.: Understanding the Linux Kernel, 3rd edn. O’Reilly (2005)Google Scholar
  27. 27.
    Kivity, A., Kamay, Y., Laor, D., Lublin, U., Liguori, A.: kvm: the Linux virtual machine monitor. In: Proceedings of the 2007 Ottawa Linux Symposium, pp. 225–230 (2007)Google Scholar
  28. 28.
    QEMU: (2011). Accessed 15 August 2011
  29. 29.
    SPEC CPU2000: (2011). Accessed 15 August 2011
  30. 30.
    SPECjbb2005: (2011). Accessed 15 August 2011
  31. 31.
    The Apache Software Foundation: (2011). Accessed 15 August 2011
  32. 32.
    http_load: (2011). Accessed 15 August 2011
  33. 33.
    Hwang, W., Roh, Y., Park, Y., KPark, W., Park, K.H.: HyperDealer: reference-pattern-aware instant memory balancing for consolidated virtual machines. In: Proceeding of the 2010 IEEE 3rd International Conference on Cloud Computing, pp. 426–434 (2010)Google Scholar
  34. 34.
    Milos, G., Myrray, D.G., Hand, S., Fetterman, M.A.: Satori: enlightened page sharing. In: Proceedings of the 2009 Conference on USENIX Annual Technical Conference (2009)Google Scholar
  35. 35.
    Wong, T.M., Wilkes, J.: My cache or yours? Making storage more exclusive. In: Proceedings of the USENIX Annual Technical Conference, pp. 161–175. Monterey, CA (2002)Google Scholar
  36. 36.
    Chen, Z., Zhou, Y., Li, K.: Eviction based placement for storage caches. In: Proceedings of the USENIX Annual Technical Conference, pp. 269–282. San Antonio, TX (2003)Google Scholar
  37. 37.
    Milojicic, D.S., Douglis, F., Paindaveine, Y., Zhou, S.: Process Migration, ACM Comput. Surv. 32(3), 241–299 (2010)CrossRefGoogle Scholar
  38. 38.
    Kozuch, M., Satyanarayanan, M.: Internet suspend/ resume. In: Proceedings of the 4th IEEE Workshop on Mobile Computing Systems and Applications, pp. 40–46 (2002)Google Scholar

Copyright information

© Springer Science+Business Media B.V. 2012

Authors and Affiliations

  • Changwoo Min
    • 1
    • 2
  • Inhyeok Kim
    • 1
  • Taehyoung Kim
    • 1
  • Young Ik Eom
    • 1
  1. 1.School of Information & Communication EngineeringSungkyunkwan UniversityGyeonggi-DoKorea
  2. 2.Samsung ElectronicsGyeonggi-DoKorea

Personalised recommendations