Abstract
Learning-based anomaly detection systems build models of the expected behavior of applications by analyzing events that are generated during their normal operation. Once these models have been established, subsequent events are analyzed to identify deviations, given the assumption that anomalies usually represent evidence of an attack.
Host-based anomaly detection systems often rely on system call traces to build models and perform intrusion detection. Recently, these systems have been criticized, and it has been shown how detection can be evaded by executing an attack using a carefully crafted exploit. This weakness is caused by the fact that existing models do not take into account all available features of system calls. In particular, some attacks will go undetected because the models do not make use of system call arguments. To solve this problem, we have developed an anomaly detection technique that utilizes the information contained in these parameters. Based on our approach, we developed a host-based intrusion detection system that identifies attacks using a composition of various anomaly metrics.
This paper presents our detection techniques and the tool based on them. The experimental evaluation shows that it is possible to increase both the effectiveness and the precision of the detection process compared to previous approaches. Nevertheless, the system imposes only minimal overhead.
Chapter PDF
Similar content being viewed by others
References
Advisory: Input validation problems in wuftpd (2000), http://www.cert.org/advisories/CA-2000-13.html
Advisory: Buffer overflow in linuxconf (2002), http://www.idefense.com/advisory/08.28.02.txt
Bernaschi, M., Gabrielli, E., Mancini, L.V.: REMUS: a Security-Enhanced Operating System. ACM Transactions on Information and System Security 5(36) (February 2002)
Billingsley, P.: Probability and Measure, 3rd edn. Wiley-Interscience, Hoboken (April 1995)
Chari, S.N., Cheng, P.-C.: Bluebox: A policy-driven, host-based intrusion detection system. In: Proceedings of the 2002 ISOC Symposium on Network and Distributed System Security (NDSS 2002), San Diego, CA (2002)
Denning, D.E.: An Intrusion Detection Model. IEEE Transactions on Software Engineering 13(2), 222–232 (1987)
Eckmann, S.T., Vigna, G., Kemmerer, R.A.: STATL: An Attack Language for State-based Intrusion Detection. Journal of Computer Security 10(1/2), 71–104 (2002)
Forrest, S.: A Sense of Self for UNIX Processes. In: Proceedings of the IEEE Symposium on Security and Privacy, Oakland, CA, pp. 120–128 (May 1996)
Ghosh, A.K., Wanken, J., Charron, F.: Detecting Anomalous and Unknown Intrusions Against Programs. In: Proceedings of the Annual Computer Security Application Conference (ACSAC 1998), Scottsdale, AZ, pp. 259–267 (December 1998)
Goldberg, I., Wagner, D., Thomas, R., Brewer, E.A.: A secure environment for untrusted helper applications. In: Proceedings of the 6th Usenix Security Symposium, San Jose, CA, USA (1996)
Javitz, H.S., Valdes, A.: The SRI IDES Statistical Anomaly Detector. In: Proceedings of the IEEE Symposium on Security and Privacy (May 1991)
Ko, C., Ruschitzka, M., Levitt, K.: Execution Monitoring of Security-Critical Programs in Distributed Systems: A Specification-based Approach. In: Proceedings of the 1997 IEEE Symposium on Security and Privacy, pp. 175–187 (May 1997)
MIT Lincoln Laboratory. DARPA Intrusion Detection Evaluation (1999), http://www.ll.mit.edu/IST/ideval/
Lee, S.Y., Low, W.L., Wong, P.Y.: Learning Fingerprints for a Database Intrusion Detection System. In: Gollmann, D., Karjoth, G., Waidner, M. (eds.) ESORICS 2002. LNCS, vol. 2502, p. 264. Springer, Heidelberg (2002)
Lee, W., Stolfo, S., Chan, P.: Learning Patterns from Unix Process Execution Traces for Intrusion Detection. In: Proceedings of the AAAI Workshop: AI Approaches to Fraud Detection and Risk Management (July 1997)
Lee, W., Stolfo, S., Mok, K.: Mining in a Data-flow Environment: Experience in Network Intrusion Detection. In: Proceedings of the 5th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD 1999), San Diego, CA (August 1999)
Lindqvist, U., Porras, P.A.: Detecting Computer and Network Misuse with the Production-Based Expert System Toolset (P-BEST). In: IEEE Symposium on Security and Privacy, Oakland, California, pp. 146–161 (May 1999)
Paxson, V.: Bro: A System for Detecting Network Intruders in Real-Time. In: Proceedings of the 7th USENIX Security Symposium, San Antonio, TX (January 1998)
Provos, N.: Improving host security with system call policies. In: Proceedings of the 12th Usenix Security Symposium, Washington, DC (2003)
SNARE - System iNtrusion Analysis and Reporting Environment, http://www.intersectalliance.com/projects/Snare
Stolcke, A., Omohundro, S.: HiddenMarkov Model Induction by Bayesian Model Merging. In: Advances in Neural Information Processing Systems (1993)
Stolcke, A., Omohundro, S.: Inducing probabilistic grammars by bayesian model merging. In: International Conference on Grammatical Inference (1994)
Tan, K., Maxion, R.: “Why 6?” Defining the Operational Limits of Stide, an Anomaly-Based Intrusion Detector. In: Proceedings of the IEEE Symposium on Security and Privacy, Oakland, CA, pp. 188–202 (May 2002)
Tan, K.M.C., Killourhy, K.S., Maxion, R.A.: Undermining an Anomaly-Based Intrusion Detection System Using Common Exploits. In: Proceedings of the 5th International Symposium on Recent Advances in Intrusion Detection, Zurich, Switzerland, pp. 54–73 (October 2002)
Wagner, D., Dean, D.: Intrusion Detection via Static Analysis. In: Proceedings of the IEEE Symposium on Security and Privacy, Oakland, CA. IEEE Press, Los Alamitos (May 2001)
Wagner, D., Soto, P.: Mimicry Attacks on Host-Based Intrusion Detection Systems. In: Proceedings of the 9th ACM Conference on Computer and Communications Security, Washington DC, USA, pp. 255–264 (November 2002)
Warrender, C., Forrest, S., Pearlmutter, B.A.: Detecting intrusions using system calls: Alternative data models. In: IEEE Symposium on Security and Privacy, pp. 133–145 (1999)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kruegel, C., Mutz, D., Valeur, F., Vigna, G. (2003). On the Detection of Anomalous System Call Arguments. In: Snekkenes, E., Gollmann, D. (eds) Computer Security – ESORICS 2003. ESORICS 2003. Lecture Notes in Computer Science, vol 2808. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-39650-5_19
Download citation
DOI: https://doi.org/10.1007/978-3-540-39650-5_19
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-20300-1
Online ISBN: 978-3-540-39650-5
eBook Packages: Springer Book Archive