Behavioral Distance for Intrusion Detection

  • Debin Gao
  • Michael K. Reiter
  • Dawn Song
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3858)

Abstract

We introduce a notion, behavioral distance, for evaluating the extent to which processes—potentially running different programs and executing on different platforms—behave similarly in response to a common input. We explore behavioral distance as a means to detect an attack on one process that causes its behavior to deviate from that of another. We propose a measure of behavioral distance and a realization of this measure using the system calls emitted by processes. Through an empirical evaluation of this measure using three web servers on two different platforms (Linux and Windows), we demonstrate that this approach holds promise for better intrusion detection with moderate overhead.

Keywords

Intrusion detection system call behavioral distance 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
  2. 2.
    Alvisi, L., Malkhi, D., Pierce, E., Reiter, M.K.: Fault detection for Byzantine quorum systems. IEEE Transactions on Parallel Distributed Systems 12(9) (September 2001)Google Scholar
  3. 3.
    Buskens, R.W., Bianchini Jr., R.P.: Distributed on-line diagnosis in the presence of arbitrary faults. In: Proceedings of the 23rd International Symposium on Fault-Tolerant Computing, June 1993, pp. 470–479 (1993)Google Scholar
  4. 4.
    Castro, M., Rodrigues, R., Liskov, B.: Base: Using abstraction to improve fault tolerance. ACM Transactions on Computer Systems (TOCS) 21(3), 236–269 (2003)CrossRefGoogle Scholar
  5. 5.
    Chen, L., Avizienes, A.: n-version programming: A fault-tolerance approach to reliability of software operation. In: Proceedings of the 8th International Symposium on Fault-Tolerant Computing, pp. 3–9 (1978)Google Scholar
  6. 6.
    Cheung, S., Crawford, R., Dilger, M., Frank, J., Hoagland, J., Levitt, K., Rowe, J., Staniford-Chen, S., Yip, R., Zerkle, D.: The design of GrIDS: A graph-based intrusion detection system. Technical Report CSE-99-2, Computer Science Department, U.C. Davis (1999)Google Scholar
  7. 7.
    Collberg, C., Thomborson, C., Low, D.: Manufacturing cheap, resilient, and stealthy opaque constructs. In: Proceedings of the ACM Symposium on Principles of Programming Languages (January 1998)Google Scholar
  8. 8.
    Feng, H.H., Giffin, J.T., Huang, Y., Jha, S., Lee, W., Miller, B.P.: Formalizing sensitivity in static analysis for intrusion detection. In: Proceedings of the 2004 IEEE Symposium on Security and Privacy (2004)Google Scholar
  9. 9.
    Feng, H.H., Kolesnikov, O.M., Fogla, P., Lee, W., Gong, W.: Anomaly detection using call stack information. In: Proceedings of the 2003 IEEE Symposium on Security and Privacy (2003)Google Scholar
  10. 10.
    Forrest, S., Langstaff, T.A.: A sense of self for unix processes. In: Proceedings of the 1996 IEEE Symposium on Security and Privacy (1996)Google Scholar
  11. 11.
    The Apache Software Foundation. Apache http server, http://httpd.apache.org
  12. 12.
    Gao, D., Reiter, M.K., Song, D.: Gray-box extraction of execution graph for anomaly detection. In: Proceedings of the 11th ACM Conference on Computer & Communication Security (2004)Google Scholar
  13. 13.
    Gao, D., Reiter, M.K., Song, D.: On gray-box program tracking for anomaly detection. In: Proceedings of the 13th USENIX Security Symposium (2004)Google Scholar
  14. 14.
    Giffin, J.T., Jha, S., Miller, B.P.: Detecting manipulated remote call streams. In: Proceedings of the 11th USENIX Security Symposium (2002)Google Scholar
  15. 15.
    Giffin, J.T., Jha, S., Miller, B.P.: Efficient context-sensitive intrusion detection. In: Proceedings of Symposium on Network and Distributed System Security (2004)Google Scholar
  16. 16.
    Kruegel, C., Mutz, D., Valeur, F., Vigna, G.: On the detection of anomalous system call arguments. In: Snekkenes, E., Gollmann, D. (eds.) ESORICS 2003. LNCS, vol. 2808, pp. 326–343. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  17. 17.
    Lamport, L.: The implementation of reliable distributed multiprocess systems. Computer Networks 2 (1978)Google Scholar
  18. 18.
    Lu, X.: A Linux executable editing library. Master’s thesis, Computer and Information Science Department, National Unviersity of Singpaore (1999)Google Scholar
  19. 19.
    Nebbett, G.: Windows NT/2000 Native API Reference. Sams Publishing (2000)Google Scholar
  20. 20.
    Nei, M., Kumar, S.: Molecular Evolution and Phylogenetics. Oxford University Press, Oxford (2000)Google Scholar
  21. 21.
    Ning, P., Cui, Y., Reeves, D.S.: Analyzing intensive intrusion alerts via correlation. In: Wespi, A., Vigna, G., Deri, L. (eds.) RAID 2002. LNCS, vol. 2516, p. 74. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  22. 22.
    Prasad, M., Chiueh, T.: A binary rewriting defense against stack based buffer overflow attacks. In: Proceedings of the USENIX Annual Technical Conference (June 2003)Google Scholar
  23. 23.
    Rigoutsos, I., Floratos, A.: Combinatorial pattern discovery in biological sequences. Bioinformatics 14(1), 55–67 (1998)CrossRefGoogle Scholar
  24. 24.
    Romer, T., Voelker, G., Lee, D., Wolman, A., Wong, W., Levy, H., Bershad, B., Chen, B.: Instrumentation and optimization of win32/intel executables using etch. In: Proceeding of the USENIX Windows NT Workshop (August 1997)Google Scholar
  25. 25.
    Schneider, F.B.: Implementing fault-tolerant services using the state machine approach: A tutorial. ACM Computing Surveys 22(4), 299–319 (1990)CrossRefGoogle Scholar
  26. 26.
    Schwarz, B., Debray, S., Andrews, G.: Disassembly of executable code revisited. In: Proceeding of the Working Conference on Reverse Engineering, pp. 45–54 (2002)Google Scholar
  27. 27.
    Sekar, R., Bendre, M., Dhurjati, D., Bollineni, P.: A fast automaton-based method for detecting anomalous program behaviors. In: Proceedings of the 2001 IEEE Symposium on Security and Privacy (2001)Google Scholar
  28. 28.
    Sellers, P.H.: On the theory and computation of evolutionary distances. SIAM J. Appl. Math. 26, 787–793Google Scholar
  29. 29.
    Shin, K., Ramanathan, P.: Diagnosis of processors with Byzantine faults in a distributed computing system. In: Proceedings of the 17th International Symposium on Fault-Tolerant Computing, pp. 55–60 (1987)Google Scholar
  30. 30.
    Snapp, S.R., Smaha, S.E., Teal, D.M., Grance, T.: The DIDS (Distributed Intrusion Detection System) prototype. In: Proceedings of the Summer USENIX Conference, pp. 227–233 (1992)Google Scholar
  31. 31.
    Tan, K., McHugh, J., Killourhy, K.: Hiding intrusions: From the abnormal to the normal and beyond. In: Proceedings of the 5th International Workshop on Information Hiding (October 2002)Google Scholar
  32. 32.
    Aprelium Technologies. Abyss web server, http://www.aprelium.com
  33. 33.
    Valdes, A., Skinner, K.: Probabilistic alert correlation. In: Lee, W., Mé, L., Wespi, A. (eds.) RAID 2001. LNCS, vol. 2212, p. 54. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  34. 34.
  35. 35.
    Wagner, D., Dean, D.: Intrusion detection via static analysis. In: Proceedings of the 2001 IEEE Symposium on Security and Privacy (2001)Google Scholar
  36. 36.
    Wagner, D., Soto, P.: Mimicry attacks on host-based intrusion detection systems. In: Proceedings of the 9th ACM Conference on Computer and Communications Security (2002)Google Scholar
  37. 37.
    Wespi, A., Dacier, M., Debar, H.: Intrusion detection using variable-length audit trail patterns. In: Proceedings of the 2000 Recent Advances in Intrusion Detection (2000)Google Scholar
  38. 38.
    Xie, Y., Kim, H., O’Hallaron, D., Reiter, M.K., Zhang, H.: Seurat: A pointillist approach to anomaly detection. In: Jonsson, E., Valdes, A., Almgren, M. (eds.) RAID 2004. LNCS, vol. 3224, pp. 238–257. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  39. 39.
    Yin, J., Martin, J.-P., Venkataramani, A., Alvisi, L., Dahlin, M.: Separating agreement from execution for Byzantine fault tolerant services. In: Proceedings of the 19th ACM Symposium on Operating System Principles (2003)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Debin Gao
    • 1
  • Michael K. Reiter
    • 2
  • Dawn Song
    • 2
  1. 1.Electrical & Computer Engineering DepartmentCarnegie Mellon UniversityPittsburghUSA
  2. 2.Electrical & Computer Engineering Department, Computer Science Department, and CyLabCarnegie Mellon UniversityPittsburghUSA

Personalised recommendations