Chapter

Computer Security – ESORICS 2003

Volume 2808 of the series Lecture Notes in Computer Science pp 326-343

On the Detection of Anomalous System Call Arguments

  • Christopher KruegelAffiliated withReliable Software Group, Department of Computer Science, University of California
  • , Darren MutzAffiliated withReliable Software Group, Department of Computer Science, University of California
  • , Fredrik ValeurAffiliated withReliable Software Group, Department of Computer Science, University of California
  • , Giovanni VignaAffiliated withReliable Software Group, Department of Computer Science, University of California

* Final gross prices may vary according to local VAT.

Get Access

Abstract

Learning-based anomaly detection systems build models of the expected behavior of applications by analyzing events that are generated during their normal operation. Once these models have been established, subsequent events are analyzed to identify deviations, given the assumption that anomalies usually represent evidence of an attack.

Host-based anomaly detection systems often rely on system call traces to build models and perform intrusion detection. Recently, these systems have been criticized, and it has been shown how detection can be evaded by executing an attack using a carefully crafted exploit. This weakness is caused by the fact that existing models do not take into account all available features of system calls. In particular, some attacks will go undetected because the models do not make use of system call arguments. To solve this problem, we have developed an anomaly detection technique that utilizes the information contained in these parameters. Based on our approach, we developed a host-based intrusion detection system that identifies attacks using a composition of various anomaly metrics.

This paper presents our detection techniques and the tool based on them. The experimental evaluation shows that it is possible to increase both the effectiveness and the precision of the detection process compared to previous approaches. Nevertheless, the system imposes only minimal overhead.

Keywords

Intrusion detection anomaly models system calls