Abstract
Kernel rootkits, as one of the most elusive types of malware, pose significant challenges for investigation and defense. Among the most notable are persistent kernel rootkits, a special type of kernel rootkits that implant persistent kernel hooks to tamper with the kernel execution to hide their presence. To defend against them, an effective approach is to first identify those kernel hooks and then protect them from being manipulated by these rootkits. In this paper, we focus on the first step by proposing a systematic approach to identify those kernel hooks. Our approach is based on two key observations: First, rootkits by design will attempt to hide its presence from all running rootkit-detection software including various system utility programs (e.g., ps and ls). Second, to manipulate OS kernel control-flows, persistent kernel rootkits by their nature will implant kernel hooks on the corresponding kernel-side execution paths invoked by the security programs. In other words, for any persistent kernel rootkit, either it is detectable by a security program or it has to tamper with one of the kernel hooks on the corresponding kernel-side execution path(s) of the security program. As a result, given an authentic security program, we only need to monitor and analyze its kernel-side execution paths to identify the related set of kernel hooks that could be potentially hijacked for evasion. We have built a proof-of-concept system called HookMap and evaluated it with a number of Linux utility programs such as ls, ps, and netstat in RedHat Fedora Core 5. Our system found that there exist 35 kernel hooks in the kernel-side execution path of ls that can be potentially hijacked for manipulation (e.g., for hiding files). Similarly, there are 85 kernel hooks for ps and 51 kernel hooks for netstat, which can be respectively hooked for hiding processes and network activities. A manual analysis of eight real-world rootkits shows that our identified kernel hooks cover all those used in them.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
The adore Rootkit, http://lwn.net/Articles/75990/
The Hideme Rootkit, http://www.sophos.com/security/analyses/viruses-and-spyware/trojhidemea.html
The Strange Decline of Computer Worms, http://www.theregister.co.uk/2005/03/17/f-secure_websec/print.html
VMware, http://www.vmware.com/
Agrawal, H., Horgan, J.R.: Dynamic Program Slicing. In: Proceedings of ACM SIGPLAN 1990 Conference on Programming Language Design and Implementation (1990)
Bellard, F.: QEMU, a Fast and Portable Dynamic Translator. In: Proc. of USENIX Annual Technical Conference 2005 (FREENIX Track) (July 2005)
Butler, J.: R2̂: The Exponential Growth of Rootkit Techniques, http://www.blackhat.com/presentations/bh-usa-06/BH-US-06-Butler.pdf
Butler, J.: VICE 2.0, http://www.infosecinstitute.com/blog/README_VICE.txt
Chen, S., Xu, J., Sezer, E.C., Gauriar, P., Iyer, R.: Non-Control-Data Attacks Are Realistic Threats. In: Proc. USENIX Security Symposium (August 2005)
Grizzard, J.B.: Towards Self-Healing Systems: Re-Establishing Trust in Compromised Systems. Ph.D. thesis, Georgia Institute of Technology (May 2006)
Jiang, X., Wang, X.: “Out-of-the-Box” Monitoring of VM-Based High-Interaction Honeypots. In: Kruegel, C., Lippmann, R., Clark, A. (eds.) RAID 2007. LNCS, vol. 4637, pp. 198–218. Springer, Heidelberg (2007)
Jiang, X., Wang, X., Xu, D.: “Out-of-the-Box” Semantic View Reconstruction. In: Proceedings of the 14th ACM Conference on Computer and Communications Security (CCS 2007) (October 2007)
Petroni, N., Fraser, T., Walters, A., Arbaugh, W.: An Architecture for Specification-Based Detection of Semantic Integrity Violations in Kernel Dynamic Data. In: Proc. of the 15th USENIX Security Symposium (August 2006)
Petroni, N., Hicks, M.: Automated Detection of Persistent Kernel Control-Flow Attacks. In: Proc. of ACM CCS 2007 (October 2007)
Petroni, N.L., Fraser, T., Molina, J., Arbaugh, W.A.: Copilot - a Coprocessor-based Kernel Runtime Integrity Monitor. In: Proc. of the 13th USENIX Security Symposium (August 2004)
PJF. IceSword, http://www.antirootkit.com/software/IceSword.htm , http://pjf.blogcn.com/
Rutkowska, J.: System Virginity Verifier, http://invisiblethings.org/papers/hitb05_virginity_verifier.ppt
Rutkowska, J.: Rootkits vs. Stealth by Design Malware, http://invisiblethings.org/papers/rutkowska_bheurope2006.ppt
sd.: Linux on-the-fly kernel patching without LKM. Phrack 11(58), article 7 of 15 (2001)
Seshadri, A., Luk, M., Qu, N., Perrig, A.: SecVisor: A Tiny Hypervisor to Guarantee Lifetime Kernel Code Integrity for Commodity OSes. In: Proc. of the ACM SOSP 2007 (October 2007)
Wang, Y., Beck, D., Vo, B., Roussev, R., Verbowski, C.: Detecting Stealth Software with Strider GhostBuster. In: Proc. of the 2005 International Conference on Dependable Systems and Networks (June 2005)
Wilhelm, J., Chiueh, T.-c.: A Forced Sampled Execution Approach to Kernel Rootkit Identification. In: Kruegel, C., Lippmann, R., Clark, A. (eds.) RAID 2007. LNCS, vol. 4637, pp. 219–235. Springer, Heidelberg (2007)
Yin, H., Liang, Z., Song, D.: HookFinder: Identifying and Understanding Malware Hooking Behaviors. In: Proc. of ISOC NDSS 2008 (February 2008)
Zhang, X., Gupta, R., Zhang, Y.: Precise Dynamic Slicing Algorithms. In: Proc. of the IEEE/ACM International Conference on Software Engineering (May 2003)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Wang, Z., Jiang, X., Cui, W., Wang, X. (2008). Countering Persistent Kernel Rootkits through Systematic Hook Discovery. In: Lippmann, R., Kirda, E., Trachtenberg, A. (eds) Recent Advances in Intrusion Detection. RAID 2008. Lecture Notes in Computer Science, vol 5230. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-87403-4_2
Download citation
DOI: https://doi.org/10.1007/978-3-540-87403-4_2
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-87402-7
Online ISBN: 978-3-540-87403-4
eBook Packages: Computer ScienceComputer Science (R0)