ShadowMonitor: An Effective In-VM Monitoring Framework with Hardware-Enforced Isolation

  • Bin Shi
  • Lei CuiEmail author
  • Bo Li
  • Xudong Liu
  • Zhiyu Hao
  • Haiying Shen
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11050)


Virtual machine introspection (VMI) is one compelling technique to enhance system security in clouds. It is able to provide strong isolation between untrusted guests and security tools placed in guests, thereby enabling dependability of the security tools even if the guest has been compromised. Due to this benefit, VMI has been widely used for cloud security such as intrusion detection, security monitoring, and tampering forensics. However, existing VMI solutions suffer significant performance degradation mainly due to the high overhead upon frequent memory address translations and context-switches. This drawback limits its usage in many real-world scenarios, especially when fine-grained monitoring is desired. In this paper, we present ShadowMonitor, an effective VMI framework that enables efficient in-VM monitoring without imposing significant overhead. ShadowMonitor decomposes the whole monitoring system into two compartments and then assigns each compartment with isolated address space. By placing the monitored components in the protected compartment, ShadowMonitor guarantees the safety of both monitoring tools and guests. In addition, ShadowMonitor employs hardware-enforced instructions to design the gates across two compartments, thereby providing efficient switching between compartments. We have implemented ShadowMonitor on QEMU/KVM exploiting several hardware virtualization features. The experimental results show that ShadowMonitor could prevent several types of attacks and achieves 10\(\times \) speedup over the existing method in terms of both event monitoring and overall application performance.


Virtual machine introspection Monitor Isolation 



We would like to acknowledge all the anonymous reviewers and Dr. Manuel Egele for their valuable comments and helps in improving this paper. This work is supported by the Chinese National Key Research and Development Program (2016YFB1000103), Chinese National Natural Science Foundation of China (grant no. 61602465), U.S. NSF grants OAC-1724845, ACI-1719397, CNS-1733596, and Microsoft Research Faculty Fellowship 8300751. This work is also supported by Beijing Brain Inspired Computing Program in BCBD innovation center. Lei Cui is the corresponding author of this paper.


  1. 1.
    AMD64 architecture programmers manualGoogle Scholar
  2. 2.
    Intel 64 and IA-32 architectures software developers manualGoogle Scholar
  3. 3.
  4. 4.
  5. 5.
  6. 6.
    Xiang, G., Jin, H., Zou, D., Zhang, X., Wen, S., Zhao, F.: Vmdriver: a driver-based monitoring mechanism for virtualization. In: 29th IEEE Symposium on Reliable Distributed Systems (SRDS 2010) (2010)Google Scholar
  7. 7.
    Garfinkel, T., Rosenblum, M.: A virtual machine introspection based architecture for intrusion detection. In: The Network and Distributed System Security Symposium, NDSS 2003 (2003)Google Scholar
  8. 8.
    Carbone, M., Conover, M., Montague, B., Lee, W.: Secure and robust monitoring of virtual machines through guest-assisted introspection. In: Balzarotti, D., Stolfo, S.J., Cova, M. (eds.) RAID 2012. LNCS, vol. 7462, pp. 22–41. Springer, Heidelberg (2012). Scholar
  9. 9.
    Criswell, J., et al.: Kcofi: complete control-flow integrity for commodity operating system kernels. In: 2014 IEEE Symposium on Security and Privacy, SP 2014 (2014)Google Scholar
  10. 10.
    Criswell, J., et al.: Virtual ghost: protecting applications from hostile operating systems. In: Proceedings of ASPLOS 2014, pp. 81–96. ACM (2014).
  11. 11.
    Dolan, B., et al.: Tappan zee (north) bridge: mining memory accesses for introspection. In: Conference on Computer and Communications Security, CCS 2013 (2013)Google Scholar
  12. 12.
    Dolan-Gavitt, B., et al.: Virtuoso: narrowing the semantic gap in virtual machine introspection. In: 32nd IEEE Symposium on Security and Privacy, S&P 2011 (2011)Google Scholar
  13. 13.
    Fu, Y., Lin, Z.: Space traveling across VM: automatically bridging the semantic gap in virtual machine introspection via online kernel data redirection. In: IEEE Symposium on Security and Privacy, SP 2012 (2012)Google Scholar
  14. 14.
    Gu, Z., et al.: Process implanting: a new active introspection framework for virtualization. In: IEEE Symposium on Reliable Distributed Systems (SRDS 2011) (2011)Google Scholar
  15. 15.
    Jain, B., et al.: Sok: introspections on trust and the semantic gap. In: IEEE Symposium on Security and Privacy, SP 2014, Berkeley, CA, USA (2014)Google Scholar
  16. 16.
    Jang, D., et al.: Atra: address translation redirection attack against hardware-based external monitors. In: Proceedings of CCS 2014 (2014)Google Scholar
  17. 17.
    Jiang, X., Wang, X., Xu, D.: Stealthy malware detection and monitoring through VMM-based “out-of-the-box” semantic view reconstruction. ACM Trans. Inf. Syst. Secur. 13(2), 12:1–12:28 (2010). Scholar
  18. 18.
    Kelem, N.L., Feiertag, R.J.: A separation model for virtual machine monitors. In: IEEE Symposium on Security and Privacy, pp. 78–86 (1991).
  19. 19.
    Kwon, Y., et al.: Sego: pervasive trusted metadata for efficiently verified untrusted system services. In: Proceedings of ASPLOS 2016, pp. 277–290. ACM (2016).
  20. 20.
    Lee, H., et al.: KI-Mon: a hardware-assisted event-triggered monitoring platform for mutable kernel object. In: The 22th USENIX Security Symposium (2013)Google Scholar
  21. 21.
    Lengyel, T.K., et al.: Scalability, fidelity and stealth in the DRAKVUF dynamic malware analysis system. In: Proceedings of ACSAC 2014 (2014)Google Scholar
  22. 22.
    Liu, Y., et al.: Thwarting memory disclosure with efficient hypervisor-enforced intra-domain isolation. In: Proceedings CCS 2015, 12–16 October 2015Google Scholar
  23. 23.
    Liu, Z., et al.: CPU transparent protection of OS kernel and hypervisor integrity with programmable DRAM. In: Proceedings of ISCA 2013, 23–27 June 2013Google Scholar
  24. 24.
    Madnick, S.E., Donovan, J.J.: Application and analysis of the virtual machine approach to information system security and isolation. In: Proceedings of the Workshop on Virtual Computer Systems. ACM, New York (1973).
  25. 25.
    McKeen, F., et al.: Innovative instructions and software model for isolated execution. In: Proceedings of HASP 2013, p. 10. ACM (2013).
  26. 26.
    Moon, H., et al.: Vigilare: toward snoop-based kernel integrity monitor. In: The ACM Conference on Computer and Communications Security, CCS 2012 (2012)Google Scholar
  27. 27.
    Payne, B.D.: Simplifying virtual machine introspection using LibVMI.
  28. 28.
    Payne, B.D., Lee, W.: Secure and flexible monitoring of virtual machines. In: 23rd Annual Computer Security Applications Conference (ACSAC 2007), 10–14 December 2007, Miami Beach, Florida, USA (2007)Google Scholar
  29. 29.
    Payne, B.D., et al.: Lares: an architecture for secure active monitoring using virtualization. In: 2008 IEEE Symposium on Security and Privacy (S&P 2008) (2008)Google Scholar
  30. 30.
    Sharif, M.I., et al.: Secure in-VM monitoring using hardware virtualization. In: The Conference on Computer and Communications Security, CCS 2009 (2009)Google Scholar
  31. 31.
    Srinivasan, D., et al.: Process out-grafting: an efficient “out-of-VM" approach for fine-grained process execution monitoring. In: Proceedings of CCS 2011 (2011)Google Scholar
  32. 32.
    Walters, A.: The volatility framework: volatile memory artifact extraction utility framework (2007)Google Scholar
  33. 33.
    Wu, R., et al.: System call redirection: a practical approach to meeting real-world virtual machine introspection needs. In: 44th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, DSN 2014 (2014)Google Scholar
  34. 34.
    Zhao, S., et al.: Seeing through the same lens: introspecting guest address space at native speed. In: 26th USENIX Security Symposium, USENIX Security 2017 (2017)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Bin Shi
    • 1
  • Lei Cui
    • 2
    Email author
  • Bo Li
    • 1
  • Xudong Liu
    • 1
  • Zhiyu Hao
    • 2
  • Haiying Shen
    • 3
  1. 1.State Key Laboratory of Software Development EnvironmentBeihang UniversityBeijingChina
  2. 2.Institute of Information EngineeringChinese Academy of SciencesBeijingChina
  3. 3.Department of Computer ScienceUniversity of VirginiaCharlottesvilleUSA

Personalised recommendations