Integrity Checking of Function Pointers in Kernel Pools via Virtual Machine Introspection

  • Irfan Ahmed
  • Golden G. RichardIII
  • Aleksandar Zoranic
  • Vassil Roussev
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7807)

Abstract

With the introduction of kernel integrity checking mechanisms in modern operating systems, such as PatchGuard on Windows OS, malware developers can no longer easily install stealthy hooks in kernel code and well-known data structures. Instead, they must target other areas of the kernel, such as the heap, which stores a large number of function pointers that are potentially prone to malicious exploits. These areas of kernel memory are currently not monitored by kernel integrity checkers.

We present a novel approach to monitoring the integrity of Windows kernel pools, based entirely on virtual machine introspection, called HookLocator. Unlike prior efforts to maintain kernel integrity, our implementation runs entirely outside the monitored system, which makes it inherently more difficult to detect and subvert. Our system also scales easily to protect multiple virtualized targets. Unlike other kernel integrity checking mechanisms, HookLocator does not require the source code of the operating system, complex reverse engineering efforts, or the debugging map files. Our empirical analysis of kernel heap behavior shows that integrity monitoring needs to focus only on a small fraction of it to be effective; this allows our prototype to provide effective real-time monitoring of the protected system.

Keywords

Virtual machine introspection Malware Operating systems 

References

  1. 1.
    Hoglund, G., Butler, J.: Rootkits: Subverting the Windows Kernel, 1st edn. Addison-Wesley Professional, Upper Saddle River (2005)Google Scholar
  2. 2.
    Yin, H., Poosankam, P., Hanna, S., Song, D.: HookScout: proactive binary-centric hook detection. In: Kreibich, C., Jahnke, M. (eds.) DIMVA 2010. LNCS, vol. 6201, pp. 1–20. Springer, Heidelberg (2010) CrossRefGoogle Scholar
  3. 3.
    Nick, J., Petroni, L., Hicks, M.: Automated detection of persistent kernel control flow attacks. In: Proceedings of the 14th ACM Conference on Computer and Communications Security (CCS 2007), Alexandria, VA, USA, pp. 103–115 (2007)Google Scholar
  4. 4.
    Baliga, A., Ganapathy, V., Iftode, L.: Automatic inference and enforcement of kernel data structure invariants. In: Proceedings of the 24th Annual Computer Security Applications Conference (ACSAC 2008), Anaheim, California, USA, pp. 77–86 (2008)Google Scholar
  5. 5.
    Carbone, M., Cui, W., Lu, L., Lee, W., Peinado, M., Jiang, X.: Mapping kernel objects to enable systematic integrity checking. In: Proceedings of the 16th ACM Conference on Computer and Communications Security (CCS 2009), Chicago, IL, USA, pp. 555–565 (2009)Google Scholar
  6. 6.
    Wang, Z., Jiang, X., Cui, W., Ning, P.: Countering kernel rootkits with lightweight hook protection. In: Proceedings of the 16th ACM Conference on Computer and Communications Security (CCS 2009), Chicago, IL, USA, pp. 545–554 (2009)Google Scholar
  7. 7.
  8. 8.
    Russinovich, M., Solomon, D.: Windows Internals: Including Windows Server 2008 and Windows Vista, 5th edn. Microsoft Press, Redmond (2009)Google Scholar
  9. 9.
    Butler, J., Hoglund, G.: VICECatch the Hookers!, In: Black Hat USA, July 2004. http://www.blackhat.com/presentations/bh-usa-04/bh-us-04-butler/bh-us-04-butler.pdf
  10. 10.
    Rutkowska, J.: System virginity verifier: defining the roadmap for malware detection on windows systems. In: Hack in the Box Security Conference, September 2005Google Scholar
  11. 11.
    Ahmed, I., Zoranic, A., Javaid, S., Richard, G.G. III.: Mod-checker: kernel module integrity checking in the cloud environment. In: 4th International Workshop on Security in Cloud Computing (CloudSec 2012), pp. 306–313 (2012)Google Scholar
  12. 12.
  13. 13.
  14. 14.
  15. 15.
    Mandt, T.: Kernel Pool Exploitation on Windows 7. http://www.mista.nu/research/MANDT-kernelpool-PAPER.pdf
  16. 16.
    mxatone and ivanlef0u.: Stealth hooking: Another way to subvert the Windows kernel. http://www.phrack.com/issues.html?issue=65&id=4
  17. 17.
    Kortchinsky, K.: Real World Kernel Pool Exploitation. http://sebug.net/paper/Meeting-Documents/syscanhk/KernelPool.pdf
  18. 18.
    Riley, R., Jiang, X., Xu, D.: Multi-aspect proling of kernel rootkit behavior. In: The Proceedings of the 4th ACM European Conference on Computer Systems (EuroSys 2009), Nuremberg, Germany, pp. 47–60 (2009)Google Scholar
  19. 19.
    Yin, H., Liang, Z., Song, D.: HookFinder: identifying and understanding malware hooking behaviors. In: Proceedings of the 15th Annual Network and Distributed System Security Symposium (NDSS 2008), February 2008Google Scholar
  20. 20.

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Irfan Ahmed
    • 1
  • Golden G. RichardIII
    • 1
  • Aleksandar Zoranic
    • 1
  • Vassil Roussev
    • 1
  1. 1.Department of Computer ScienceUniversity of New OrleansNew OrleansUSA

Personalised recommendations