Abstract
The behavioral distance between two processes is a measure of the deviation of their behaviors. Behavioral distance has been proposed for detecting the compromise of a process, by computing its behavioral distance from another process executed on the same input. Provided that the two processes are diverse and so unlikely to fall prey to the same attacks, an increase in behavioral distance might indicate the compromise of one of them. In this paper we propose a new approach to behavioral distance calculation using a new type of Hidden Markov Model. We also empirically evaluate the intrusion detection capability of our proposal when used to measure the distance between the system-call behaviors of diverse web servers. Our experiments show that it detects intrusions with substantially greater accuracy and with performance overhead comparable to that of prior proposals.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Abd-El-Malek, M., Ganger, G.R., Goodson, G.R., Reiter, M.K., Wylie, J.J.: Fault-scalable Byzantine fault-tolerant services. In: Proceedings of the 20th ACM Symposium on Operating Systems Principles, pp. 59–74 (October 2005)
Alvisi, L., Malkhi, D., Pierce, E., Reiter, M.K.: Fault detection for Byzantine quorum systems. IEEE Transactions on Parallel Distributed Systems 12(9) (September 2001)
Baum, L.E., Petrie, T.: Statistical inference for probabilistic functions of finite state Markov chains. Ann. Math. Statist. 37, 1554–1563 (1966)
Bhatkar, S., Chaturvedi, A., Sekar, R.: Dataflow anomaly detection. In: Proceedings of the 2006 IEEE Symposium on Security and Privacy (2006)
Buskens, R.W., Bianchini Jr., R.P.: Distributed on-line diagnosis in the presence of arbitrary faults. In: Proceedings of the 23rd International Symposium on Fault-Tolerant Computing, pp. 470–479 (June 1993)
Cachin, C., Poritz, J.A.: Secure intrusion-tolerant replication on the Internet. In: Proceedings of the 2002 International Conference on Dependable Systems and Networks (2002)
Castro, M., Liskov, B.: Practical Byzantine fault tolerance and proactive recovery. ACM Transactions on Computer Systems 20(4) (November 2002)
Castro, M., Rodrigues, R., Liskov, B.: BASE: Using abstraction to improve fault tolerance. ACM Transactions on Computer Systems 21(3) (August 2003)
Chen, L., Avizienis, A.: N-version programming: A fault-tolerance approach to reliability of software operation. In: Proceedings of the 8th International Symposium on Fault-Tolerant Computing, pp. 3–9 (1978)
Cho, S.-B., Han, S.-J.: Two sophisticated techniques to improve HMM-based intrusion detection systems. In: Vigna, G., Krügel, C., Jonsson, E. (eds.) RAID 2003. LNCS, vol. 2820, pp. 207–219. Springer, Heidelberg (2003)
Cox, B., Evans, D., Filipi, A., Rowanhill, J., Hu, W., Davidson, J., Knight, J., Nguyen-Tuong, A., Hiser, J.: N-variant systems – A secretless framework for security through diversity. In: Proceedings of the 15th USENIX Security Symposium (August 2006)
Davis, R.I.A., Lovell, B.C., Caelli, T.: Improved estimation of Hidden Markov Model parameters from multiple observation sequences. In: Proceedings of the 16th International Conference on Pattern Recognition (ICPR 2002) (2002)
Feng, H.H., Giffin, J.T., Huang, Y., Jha, S., Lee, W., Miller, B.P.: Formalizing sensitivity in static analysis for intrusion detection. In: Proceedings of the 2004 IEEE Symposium on Security and Privacy (2004)
Feng, H.H., Kolesnikov, O.M., Fogla, P., Lee, W., Gong, W.: Anomaly detection using call stack information. In: Proceedings of the 2003 IEEE Symposium on Security and Privacy (2003)
Forrest, S., Hofmeyr, S.A., Somayaji, A., Longstaff, T.A.: A sense of self for Unix processes. In: Proceedings of the 1996 IEEE Symposium on Security and Privacy (1996)
Gao, D., Reiter, M.K., Song, D.: Gray-box extraction of execution graph for anomaly detection. In: Proceedings of the 11th ACM Conference on Computer & Communication Security (2004)
Gao, D., Reiter, M.K., Song, D.: On gray-box program tracking for anomaly detection. In: Proceedings of the 13th USENIX Security Symposium (2004)
Gao, D., Reiter, M.K., Song, D.: Behavioral distance for intrusion detection. In: Valdes, A., Zamboni, D. (eds.) RAID 2005. LNCS, vol. 3858, pp. 63–81. Springer, Heidelberg (2006)
Giffin, J.T., Jha, S., Miller, B.P.: Detecting manipulated remote call streams. In: Proceedings of the 11th USENIX Security Symposium (2002)
Giffin, J.T., Jha, S., Miller, B.P.: Efficient context-sensitive intrusion detection. In: Proceedings of Symposium on Network and Distributed System Security (2004)
Krügel, C., Mutz, D., Valeur, F., Vigna, G.: On the detection of anomalous system call arguments. In: Snekkenes, E., Gollmann, D. (eds.) ESORICS 2003. LNCS, vol. 2808, pp. 326–343. Springer, Heidelberg (2003)
Lamport, L.: The implementation of reliable distributed multiprocess systems. Computer Networks 2, 95–114 (1978)
Meyer, I.M., Durbin, R.: Comparative ab initio prediction of gene structures using pair HMMs. Oxford University Press, Oxford (2002)
Pachter, L., Alexandersson, M., Cawley, S.: Applications of generalized pair Hidden Markov Models to alignment and gene finding problems. Computational Biology 9(2) (2002)
Rabiner, L.R.: A tutorial on Hidden Markov Models and selected applications in speech recognition. Proceedings of IEEE (February 1989)
Reiter, M.K.: Secure agreement protocols: Reliable and atomic group multicast in Rampart. In: Proceedings of the 2nd ACM Conference on Computer and Communication Security, pp. 68–80 (November 1994)
Schneider, F.B.: Implementing fault-tolerant services using the state machine approach: A tutorial. ACM Computing Surveys 22(4), 299–319 (1990)
Sekar, R., Bendre, M., Dhurjati, D., Bollineni, P.: A fast automaton-based method for detecting anomalous program behaviors. In: Proceedings of the 2001 IEEE Symposium on Security and Privacy (2001)
Sellers, P.H.: On the theory and computation of evolutionary distances. SIAM J. Appl. Math. 26, 787–793 (1974)
Shin, K., Ramanathan, P.: Diagnosis of processors with Byzantine faults in a distributed computing system. In: Proceedings of the 17th International Symposium on Fault-Tolerant Computing, pp. 55–60 (1987)
Tan, K., McHugh, J., Killourhy, K.: Hiding intrusions: From the abnormal to the normal and beyond. In: Petitcolas, F.A.P. (ed.) IH 2002. LNCS, vol. 2578, pp. 1–17. Springer, Heidelberg (2003)
Wagner, D., Dean, D.: Intrusion detection via static analysis. In: Proceedings of the 2001 IEEE Symposium on Security and Privacy (2001)
Wagner, D., Soto, P.: Mimicry attacks on host-based intrusion detection systems. In: Proceedings of the 9th ACM Conference on Computer and Communications Security (2002)
Warrender, C., Forrest, S., Pearlmutter, B.: Detecting intrusions using system calls: alternative data models. In: Proceedings of the 1999 IEEE Symposium on Security and Privacy (1999)
Wespi, A., Dacier, M., Debar, H.: Intrusion detection using variable-length audit trail patterns. In: Debar, H., Mé, L., Wu, S.F. (eds.) RAID 2000. LNCS, vol. 1907, p. 110. Springer, Heidelberg (2000)
Yin, J., Martin, J., Venkataramani, A., Alvisi, L., Dahlin, M.: Separating agreement from execution for Byzantine fault tolerant services. In: Proceedings of the 19th ACM Symposium on Operating System Principles (October 2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Gao, D., Reiter, M.K., Song, D. (2006). Behavioral Distance Measurement Using Hidden Markov Models. In: Zamboni, D., Kruegel, C. (eds) Recent Advances in Intrusion Detection. RAID 2006. Lecture Notes in Computer Science, vol 4219. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11856214_2
Download citation
DOI: https://doi.org/10.1007/11856214_2
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-39723-6
Online ISBN: 978-3-540-39725-0
eBook Packages: Computer ScienceComputer Science (R0)