Fast Signature Matching Using Extended Finite Automaton (XFA)

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5352)


Automata-based representations and related algorithms have been applied to address several problems in information security, and often the automata had to be augmented with additional information. For example, extended finite-state automata (EFSA) augment finite-state automata (FSA) with variables to track dependencies between arguments of system calls. In this paper, we introduce extended finite automata (XFAs) which augment FSAs with finite scratch memory and instructions to manipulate this memory. Our primary motivation for introducing XFAs is signature matching in Network Intrusion Detection Systems (NIDS). Representing NIDS signatures as deterministic finite-state automata (DFAs) results in very fast signature matching but for several types of signatures DFAs can blowup in space. Nondeterministic finite-state automata (NFA) representation of NIDS signatures results in a succinct representation but at the expense of higher time complexity for signature matching. In other words, DFAs are time-efficient but space-inefficient, and NFAs are space-efficient but time-inefficient. Our goal is to find a representation of signatures that is both time and space efficient. In our experiments we have noticed that for a large class of NIDS signatures XFAs have time complexity similar to DFAs and space complexity similar to NFAs. For our test set, XFAs use 10 times less memory than a DFA-based solution, yet achieve 20 times higher matching speeds.


Intrusion Detection Regular Expression Signature Match String Match Hybrid Automaton 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Aho, A.V., Corasick, M.J.: Efficient string matching: An aid to bibliographic search. Communications of the ACM (June 1975)Google Scholar
  2. 2.
    Alur, R.: Timed automata. In: Proceedings of the Int. Conf. on Computer Aided Verification, pp. 8–22 (1999)Google Scholar
  3. 3.
    Boyer, R.S., Moore, J.S.: A fast string searching algorithm. Communications of the ACM 20 (October 1977)Google Scholar
  4. 4.
    Brumley, D., Newsome, J., Song, D., Wang, H., Jha, S.: Towards automatic generation of vulnerability-based signatures. In: IEEE Symposium on Security and Privacy, Oakland, California (May 2006)Google Scholar
  5. 5.
    Clark, C.R., Schimmel, D.E.: Scalable pattern matching for high-speed networks. In: IEEE Symposium on Field-Programmable Custom Computing Machines (FCCM), pp. 249–257 (April 2004)Google Scholar
  6. 6.
    Coit, C.J., Staniford, S., McAlerney, J.: Towards faster pattern matching for intrusion detection or exceeding the speed of Snort. In: 2nd DARPA Information Survivability Conference and Exposition (June 2001)Google Scholar
  7. 7.
    Crosby, S.: Denial of service through regular expressions. In: Usenix Security work in progress report (August 2003)Google Scholar
  8. 8.
    Eckmann, S.T., Vigna, G., Kemmerer, R.A.: STATL: An attack language for state-based intrusion detection. Journal of Computer Security 10(1/2), 71–104 (2002)CrossRefGoogle Scholar
  9. 9.
    Fisk, M., Varghese, G.: Fast content-based packet handling for intrusion detection. TR CS2001-0670, UC San Diego (May 2001)Google Scholar
  10. 10.
    Fortnow, L.: Nondeterministic polynomial time versus nondeterministic logarthmic space: Time-space tradeoffs for satisfiability. In: Proceedings of Twelfth IEEE Conference on Computational Complexity (1997)Google Scholar
  11. 11.
    Handley, M., Paxson, V., Kreibich, C.: Network intrusion detection: Evasion, traffic normalization, and end-to-end protocol semantics. In: Usenix Security (August 2001)Google Scholar
  12. 12.
    Henzinger, T.A.: The theory of hybrid automata. In: Proceedings of the 11th Annual Symposium on Logic in Computer Science (LICS), pp. 278–292 (1996)Google Scholar
  13. 13.
    Hopcroft, J.E., Ullman, J.D.: Introduction to Automata Theory, Languages, and Computation. Addison-Wesley, Reading (1979)zbMATHGoogle Scholar
  14. 14.
    Jordan, M.: Dealing with metamorphism. Virus Bulletin Weekly (2002)Google Scholar
  15. 15.
    Kumar, S., Dharmapurikar, S., Yu, F., Crowley, P., Turner, J.: Algorithms to accelerate multiple regular expressions matching for deep packet inspection. In: Proceedings of ACM SIGCOMM (September 2006)Google Scholar
  16. 16.
    Liu, R.-T., Huang, N.-F., Chen, C.-H., Kao, C.-N.: A fast string-matching algorithm for network processor-based intrusion detection system. Transactions on Embedded Computing Sys. 3(3), 614–633 (2004)CrossRefGoogle Scholar
  17. 17.
    Narayanan, A., Shmatikov, V.: Fast dictionary attacks on passwords using time-space tradeoff. In: ACM Conference on Computer and Communications Security (CCS) (2005)Google Scholar
  18. 18.
    Oechslin, P.: Making a faster cryptanalytic time-memory trade-off. In: Boneh, D. (ed.) CRYPTO 2003. LNCS, vol. 2729. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  19. 19.
    Ptacek, T., Newsham, T.: Insertion, evasion and denial of service: Eluding network intrusion detection. In: Secure Networks, Inc. (January 1998)Google Scholar
  20. 20.
    Roesch, M.: Snort - lightweight intrusion detection for networks. In: Proceedings of the 13th Systems Administration Conference, USENIX (1999)Google Scholar
  21. 21.
    Rubin, S., Jha, S., Miller, B.: Language-based generation and evaluation of NIDS signatures. In: IEEE Symposium on Security and Privacy (May 2005)Google Scholar
  22. 22.
    Rubin, S., Jha, S., Miller, B.P.: Protomatching network traffic for high throughput network intrusion detection. In: ACM Conference on Computer and Communications Security (CCS), pp. 47–58 (2006)Google Scholar
  23. 23.
    Sekar, R., Uppuluri, P.: Synthesizing fast intrusion prevention/detection systems from high-level specifications. In: Usenix Security (August 1999)Google Scholar
  24. 24.
    Shankar, U., Paxson, V.: Active mapping: Resisting NIDS evasion without altering traffic. In: IEEE Symposium on Security and Privacy (May 2003)Google Scholar
  25. 25.
    Sidhu, R., Prasanna, V.: Fast regular expression matching using FPGAs. In: Field-Programmable Custom Computing Machines (FCCM) (April 2001)Google Scholar
  26. 26.
    Smith, R., Estan, C., Jha, S.: Xfa: Faster signature matching with extended automata. In: IEEE Symposium on Security and Privacy (2008)Google Scholar
  27. 27.
    Smith, R., Estan, C., Jha, S., Kong, S.: Deflating the big bang: fast and scalable deep packet inspection with extended finite automata. In: SIGCOMM (2008)Google Scholar
  28. 28.
    Sommer, R., Paxson, V.: Enhancing byte-level network intrusion detection signatures with context. In: ACM Conference on Computer and Communications Security (CCS) (2003)Google Scholar
  29. 29.
    Sourdis, I., Pnevmatikatos, D.: Fast, large-scale string match for a 10gbps fpga-based network intrusion detection system. In: International Conference on Field Programmable Logic and Applications (September 2003)Google Scholar
  30. 30.
    Sourdis, I., Pnevmatikatos, D.: Pre-decoded CAMs for efficient and high-speed NIDS pattern matching. In: IEEE Symposium on Field-Programmable Custom Computing Machines (FCCM) (April 2004)Google Scholar
  31. 31.
    Tan, L., Sherwood, T.: A high throughput string matching architecture for intrusion detection and prevention. In: International Symposium on Computer Architecture (ISCA) (June 2005)Google Scholar
  32. 32.
    Wang, H.J., Guo, C., Simon, D., Zugenmaier, A.: Shield: Vulnerability-driven network filters for preventing known vulnerability exploits. In: Proceedings of the 2004 ACM SIGCOMM Conference (August 2004)Google Scholar
  33. 33.
    Yegneswaran, V., Giffin, J.T., Barford, P., Jha, S.: An architecture for generating semantics-aware signatures. In: 14th USENIX Security Symposium (August 2005)Google Scholar
  34. 34.
    Yu, F., Chen, Z., Diao, Y., Lakshman, T.V., Katz, R.H.: Fast and memory-efficient regular expression matching for deep packet inspection. In: Proceedings of Architectures for Networking and Communications Systems (ANCS), pp. 93–102 (2006)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  1. 1.University of WisconsinMadisonUSA
  2. 2.Universit a di TrentoPovo-TrentoItaly

Personalised recommendations