Abstract
Rootkits are a set of software tools used by an attacker to gain unauthorized access into a system, thereby providing him with privilege to access sensitive data, conceal its own existence and allowing him to install other malicious software. An attacker needs administrative level privileges before he could install a rootkit. Rootkits are the most challenging malware to detect due to their elusive nature. Modern rootkit attacks mainly focus on modifying operating system kernel. This paper tries to provide a structured and comprehensive view of the research on rootkit detection/prevention.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Baliga, A., Ganapathy, V., Iftode, L.: Detecting kernel-level rootkits using data structure invariants. IEEE Transactions on Dependable and Secure Computing 99 (PrePrints) (2010)
Baliga, A., Iftode, L., Chen, X.: Automated containment of rootkits attacks. Computers and Security 27(7-8), 323–334 (2008), http://www.sciencedirect.com/science/article/B6V8G-4SYCPMR-1/2/0072c2079956faf503f8f683847fd3a2
Beck, D., Vo, B., Verbowski, C.: Detecting stealth software with strider ghostbuster. In: Proceedings of the 2005 International Conference on Dependable Systems and Networks, DSN 2005, pp. 368–377. IEEE Computer Society, Washington, DC, USA (2005), http://dx.doi.org/10.1109/DSN.2005.39
Bickford, J., O’Hare, R., Baliga, A., Ganapathy, V., Iftode, L.: Rootkits on smart phones: attacks, implications and opportunities. In: Proceedings of the Eleventh Workshop on Mobile Computing Systems & Applications, pp. 49–54. ACM, New York (2010)
Bratus, S., Locasto, M.E., Ramaswamy, A., Smith, S.W.: Vm-based security overkill: a lament for applied systems security research. In: Proceedings of the 2010 Workshop on New Security Paradigms, NSPW 2010, pp. 51–60. ACM, New York (2010), http://doi.acm.org/10.1145/1900546.1900554
Bunten, A.: Unix and linux based rootkits techniques and countermeasures (2004)
Clemens, J.: Intrusion Detection FAQ: Knark: Linux Kernel Subversion (2001)
Garfinkel, T., Rosenblum, M.: A virtual machine introspection based architecture for intrusion detection. In: Proc. Network and Distributed Systems Security Symposium, vol. 1, pp. 253–285. Citeseer (2003)
Jiang, X., Wang, X., Xu, D.: Stealthy malware detection through vmm-based out of the box semantic view reconstruction, pp. 128–138 (2007), http://doi.acm.org/10.1145/1315245.1315262
Kim, G.H., Spafford, E.H.: The design and implementation of tripwire: a file system integrity checker. In: Proceedings of the 2nd ACM Conference on Computer and Communications Security, CCS 1994, pp. 18–29. ACM, New York (1994), http://doi.acm.org/10.1145/191177.191183
Kroah-Hartman, G.: Signed kernel modules. Linux Journal (2004)
Kruegel, C., Robertson, W., Vigna, G.: Detecting kernel-level rootkits through binary analysis. In: Computer Security Applications Conference, Annual, pp. 91–100 (2004)
Lanzi, A., Sharif, M., Lee, W.: K-tracer: A system for extracting kernel malware behavior. In: Proceedings of the 16th Annual Network and Distributed System Security Symposium (2009)
Levine, J., Grizzard, J., Owen, H.: A methodology to detect and characterize kernel level rootkit exploits involving redirection of the system call table. In: Proceedings of Second IEEE International Information Assurance Workshop, pp. 107–125. IEEE, Los Alamitos (2005)
Levine, J.G., Grizzard, J.B., Owen, H.L.: Detecting and categorizing kernel-level rootkits to aid future detection. IEEE Security and Privacy 4, 24 (2006), http://portal.acm.org/citation.cfm?id=1115691.1115761
Lineberry, A.: Malicious Code Injection via/dev/mem. Black Hat Europe (2009), http://www.blackhat.com/presentations/bh-europe-09/Lineberry/BlackHat-Europe-2009-Lineberry-code-injection-via-dev-mem.pdf
Petroni Jr., N.L., Fraser, T., Molina, J., Arbaugh, W.A.: Copilot - a coprocessor-based kernel runtime integrity monitor. In: Proceedings of the 13th Conference on USENIX Security Symposium, SSYM 2004, vol. 13, pp. 13–13. USENIX Association, Berkeley (2004), http://portal.acm.org/citation.cfm?id=1251375.1251388
Petroni Jr., N.L., Hicks, M.: Automated detection of persistent kernel control-flow attacks. In: Proceedings of the 14th ACM Conference on Computer and Communications Security, CCS 2007, pp. 103–115. ACM, New York (2007), http://doi.acm.org/10.1145/1315245.1315260
Rhee, J., Riley, R., Xu, D., Jiang, X.: Defeating dynamic data kernel rootkit attacks via vmm-based guest-transparent monitoring. In: International Conference on Availability, Reliability and Security, pp. 74–81 (2009)
Riley, R., Jiang, X., Xu, D.: Guest-transparent prevention of kernel rootkits with vmm-based memory shadowing. In: Lippmann, R., Kirda, E., Trachtenberg, A. (eds.) RAID 2008. LNCS, vol. 5230, pp. 1–20. Springer, Heidelberg (2008)
Riley, R., Jiang, X., Xu, D.: Multi-aspect profiling of kernel rootkit behavior. In: Proceedings of the 4th ACM European Conference on Computer Systems, EuroSys 2009, pp. 47–60. ACM, New York (2009), http://doi.acm.org/10.1145/1519065.1519072
Sailer, R., Zhang, X., Jaeger, T., van Doorn, L.: Design and implementation of a tcg-based integrity measurement architecture. In: Proceedings of the 13th Conference on USENIX Security Symposium, SSYM 2004, vol. 13, p. 16. USENIX Association, Berkeley (2004), http://portal.acm.org/citation.cfm?id=1251375.1251391
Seshadri, A., Luk, M., Qu, N., Perrig, A.: Secvisor: A tiny hypervisor to provide lifetimekernel code integrity for commodity OSes. In: Proceedings of the 21st ACM Symposium on Operating Systems Principles (21st SOSP 2007), pp. 335–350. ACM SIGOPS, Stevenson (October 2007)
Wang, Z., Jiang, X., Cui, W., Ning, P.: Countering kernel rootkits with lightweight hook protection. In: Proceedings of the 16th ACM Conference on Computer and Communications Security, CCS 2009, pp. 545–554. ACM, New York (2009), http://doi.acm.org/10.1145/1653662.1653728
Wang, Z., Jiang, X., Cui, W., Wang, X.: Countering persistent kernel rootkits through systematic hook discovery. In: Lippmann, R., Kirda, E., Trachtenberg, A. (eds.) RAID 2008. LNCS, vol. 5230, pp. 21–38. Springer, Heidelberg (2008)
Wichmann, R.: A comparison of several host/file integrity monitoring programs (December 29, 2009), http://www.la-samhna.de/library/scanners.html
Yan, Q., Li, Y., Li, T., Deng, R.: Insights into Malware Detection and Prevention on Mobile Phones. In: Ślęzak, D., Kim, T.-h., Fang, W.-C., Arnett, K.P. (eds.) SecTech 2009. CCIS, vol. 58, pp. 242–249. Springer, Heidelberg (2009)
Yin, H., Liang, Z., Song, D.: Hookfinder: Identifying and understanding malware hooking behaviors. In: Proceedings of the 15th Annual Network and Distributed System Security Symposium (NDSS 2008). Citeseer (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Joy, J., John, A., Joy, J. (2011). Rootkit Detection Mechanism: A Survey. In: Nagamalai, D., Renault, E., Dhanuskodi, M. (eds) Advances in Parallel Distributed Computing. PDCTA 2011. Communications in Computer and Information Science, vol 203. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-24037-9_36
Download citation
DOI: https://doi.org/10.1007/978-3-642-24037-9_36
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-24036-2
Online ISBN: 978-3-642-24037-9
eBook Packages: Computer ScienceComputer Science (R0)