Efficiently Restoring Virtual Machines

  • Bernhard Egger
  • Erik Gustafsson
  • Changyeon Jo
  • Jeongseok Son


Saving the state of a running virtual machine (VM) for later restoration has become an indispensable tool to achieve balanced and energy-efficient usage of the underlying hardware in virtual desktop cloud environments (VDC). To free up resources, a remote user’s VM is saved to external storage when the user disconnects and restored when the user reconnects to the VDC. Existing techniques are able to reduce the size of the checkpoint image by up to 80 % by excluding duplicated memory pages; however, those techniques suffer from a significantly increased restoration time which adversely affects the deployment of the technique in VDC environments. In this paper, we introduce a method to efficiently restore VMs from such space-optimized checkpoint images. With the presented method, a VM is available to the user before the entire memory contents of the VM have been restored. Using a combination of lazy-fetch and intercepting accesses to yet unrestored pages we are able to reduce the time-to-responsiveness (TTR) for restored VMs to a few seconds. Experiments with VMs with 4 GB of memory running a wide range of benchmarks show that the proposed technique, on average, reduces the TTR by 50 % compared to the Xen hypervisor. Compared to the previously fasted restoration of space-optimized checkpoints, the proposed technique achieves a threefold speedup on average.


Virtualization Checkpointing Performance 



We thank the reviewers for the helpful and constructive feedback. This research was supported by the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Science, ICT & Future Planning (2012R1A1A1042938). ICT at Seoul National University provided research facilities for this study.


  1. 1.
    Miller, K., Pegah, M.: Virtualization: virtually at the desktop. In: Proceedings of the 35th annual ACM SIGUCCS fall conference. ACM, SIGUCCS’07, New York, NY, USA, pp. 255–260 (2007)Google Scholar
  2. 2.
    Sridharan, M., Calyam, P., Venkataraman, A., Berryman, A.: Defragmentation of resources in virtual desktop clouds for cost-aware utility-optimal allocation. In: Utility and Cloud Computing (UCC), 2011 Fourth IEEE International Conference on, pp. 253–260 (2011)Google Scholar
  3. 3.
    Citrix: Xen Desktop 7. (2013)
  4. 4.
    Laadan, O., Nieh, J.: Transparent checkpoint-restart of multiple processes on commodity operating systems. In: 2007 USENIX Annual Technical Conference on Proceedings of the USENIX Annual Technical Conference. USENIX Association, ATC’07, Berkeley, CA, USA, pp. 25:1–25:14 (2007)Google Scholar
  5. 5.
    Sancho, J.C., Petrini, F., Johnson, G., Fernandez, J., Frachtenberg, E.: On the feasibility of incremental checkpointing for scientific computing. Int. Parallel Distrib. Process. Symp. 1, 58b (2004)Google Scholar
  6. 6.
    Zhang, I., Garthwaite, A., Baskakov, Y., Barr, K.C.: Fast restore of checkpointed memory using working set estimation. In: Proceedings of the 7th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments. ACM, VEE ’11, New York, NY, USA, pp. 87–98 (2011)Google Scholar
  7. 7.
    Waldspurger, C.A.: Memory resource management in vmware esx server. SIGOPS Oper. Syst. Rev. 36, 181–194 (2002)CrossRefGoogle Scholar
  8. 8.
    Park, E., Egger, B., Lee, J.: Fast and space-efficient virtual machine checkpointing. In: Proceedings of the 7th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments. ACM, VEE’11, New York, NY, USA, pp. 75–86 (2011)Google Scholar
  9. 9.
    VirtualBox. (2013)
  10. 10.
    Koto, A., Yamada, H., Ohmura, K., Kono, K.: Towards unobtrusive vm live migration for cloud computing platforms. In: Proceedings of the Third ACM SIGOPS Asia-Pacific conference on Systems. APSys’12, Berkeley, CA, USA, USENIX Association, pp. 7–7 (2012)Google Scholar
  11. 11.
    Plank, J.S., Beck, M., Kingsley, G.: Compiler-assisted memory exclusion for fast checkpointing. IEEE Tech. Comm. Oper. Syst. Appl. Environ. 7, 10–14 (1995)Google Scholar
  12. 12.
    Heo, J., Yi, S., Cho, Y., Hong, J., Shin, S.Y.: Space-efficient page-level incremental checkpointing. In: Proceedings of the: ACM Symposium on Applied computing. ACM, SAC ’05, New York, NY, USA, pp. 1558–1562 (2005)Google Scholar
  13. 13.
    Yi, S., Heo, J., Cho, Y., Hong, J.: Adaptive page-level incremental checkpointing based on expected recovery time. In: Proceedings of the: ACM Symposium on Applied computing. ACM, SAC ’06, New York, NY, USA, pp. 1472–1476 (2006)Google Scholar
  14. 14.
    Baker, M., Sullivan, M.: The recovery box: Using fast recovery to provide high availability in the unix environment. In: Proceedings USENIX Summer Conference, pp. 31–43 (1992)Google Scholar
  15. 15.
    Li, Y., Lan, Z.: A fast restart mechanism for checkpoint, recovery protocols in networked environments. In: Dependable Systems and Networks With FTCS and DCC, 2008. DSN 2008. IEEE International Conference on, pp. 217–226 (2008)Google Scholar
  16. 16.
    Habib, I.: Virtualization with kvm. Linux J. 2008 (2008)Google Scholar
  17. 17.
  18. 18.
    Barham, P., Dragovic, B., Fraser, K., Hand, S., Harris, T., Ho, A., Neugebauer, R., Pratt, I., Warfield, A.: Xen and the art of virtualization. In: SOSP ’03: Proceedings of the Nineteenth ACM Symposium on Operating Systems Principles. ACM, New York, NY, USA, pp. 164–177 (2003)Google Scholar
  19. 19.
    Hines, M.R., Gopalan, K.: Post-copy based live virtual machine migration using adaptive pre-paging and dynamic self-ballooning. In: VEE ’09: Proceedings of the: ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments. ACM, New York, NY, USA, pp. 51–60 (2009)Google Scholar
  20. 20.
    Jiang, S., Chen, F., Zhang, X.: Clock-pro: an effective improvement of the clock replacement. In: Proceedings of the Annual Conference on USENIX Annual Technical Conference. ATEC ’05, Berkeley, CA, USA, USENIX Association, pp. 35–35 (2005)Google Scholar
  21. 21.
    The Xen Hypervisor. (2013)

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Bernhard Egger
    • 1
  • Erik Gustafsson
    • 1
  • Changyeon Jo
    • 1
  • Jeongseok Son
    • 1
  1. 1.School of Computer Science and EngineeringSeoul National UniversitySeoulKorea

Personalised recommendations