Journal of Signal Processing Systems

, Volume 51, Issue 1, pp 99–121 | Cite as

Regular Expression Matching in Reconfigurable Hardware

  • Ioannis Sourdis
  • João Bispo
  • João M. P. Cardoso
  • Stamatis Vassiliadis
Open Access
Article

Abstract

In this paper we describe a regular expression pattern matching approach for reconfigurable hardware. Following a Non-deterministic Finite Automata direction, we introduce three new basic building blocks to support constraint repetitions syntaxes more efficiently than previous works. In addition, a number of optimization techniques are employed to reduce the area cost of the designs and maximize performance. Our design methodology is supported by a tool that automatically generates the circuitry for the given regular expressions and outputs Hardware Description Language representations ready for logic synthesis. The proposed approach is evaluated on network Intrusion Detection Systems (IDS). Recent IDS use regular expressions to represent hazardous packet payload contents. They require high-speed packet processing providing a challenging case study for pattern matching using regular expressions. We use a number of IDS rulesets to show that our approach scales well as the number of regular expressions increases, and present a step-by-step optimization to survey the benefits of our techniques. The synthesis tool described in this study is used to generate hardware engines to match 300 to 1,500 IDS regular expressions using only 10–45 K logic cells and achieving throughput of 1.6–2.2 and 2.4–3.2 Gbps on Virtex2 and Virtex4 devices, respectively. Concerning the throughput per area required per matching non-Meta character, our hardware engines are 10–20× more efficient than previous Field Programmable Gate Array approaches. Furthermore, the generated designs have comparable area requirements to current application-specific integrated circuit solutions.

Keywords

regular expression pattern matching reconfigurable hardware network security 

References

  1. 1.
    S. Stephens, J. Y. Chen, M. G. Davidson, S. Thomas, and B. M. Trute, “Oracle database 10 g: a platform for blast search and regular expression pattern matching in life sciences,” Nucleic Acids Res., vol. 33 (database-Issue), 2005, pp. 675–679.CrossRefGoogle Scholar
  2. 2.
    S. Ray and M. Craven, “Learning statistical models for annotating proteins with function information using biomedical text,” BMC Bioinformatics, vol. 6, Suppl. 1, 2005, p. S:18.Google Scholar
  3. 3.
    J.-M. Champarnaud, F. Coulon, and T. Paranthoen, “Compact and fast algorithms for safe regular expression search,” Int. J. Comput. Math., vol. 81, no. 4, 2004, pp. 383–401.MATHCrossRefMathSciNetGoogle Scholar
  4. 4.
    F. Yu, Z. Chen, Y. Diao, T. V. Lakshman, and R. H. Katz, “Fast and memory-efficient regular expression matching for deep packet inspection,” in Proc. 2nd ACM/IEEE Symposium on Architectures for Networking and Communications Systems (ANCS’06), ACM Press, 2006, pp. 93–102.Google Scholar
  5. 5.
    S. Kumar, S. Dharmapurikar, F. Yu, P. Crowley, and J. Turner, “Algorithms to accelerate multiple regular expressions matching for deep packet inspection,” Comput. Commun. Rev., vol. 36, no. 4, 2006, pp. 339–350.CrossRefGoogle Scholar
  6. 6.
    F. Yu, Z. Chen, Y. Diao, T. Lakshman, and R. H. Katz, “Fast and memory-efficient regular expression matching for deep packet inspection,” EECS Department, University of California, Berkeley, Tech. Rep. UCB/EECS-2006-76, May 22 2006. [Online]. Available: http://www.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-2006-76.html.
  7. 7.
    S. Kumar, J. Turner, and J. Williams, “Advanced algorithms for fast and scalable deep packet inspection,” in Proc. of ACM/IEEE Symposium on Architecture for Networking and Sommunications Systems (ANCS’06), New York, ACM Press, 2006, pp. 81–92.Google Scholar
  8. 8.
    S. Vassiliadis, S. Wong, G. N. Gaydadjiev, K. Bertels, G. Kuzmanov, and E. M. Panainte, “The Molen polymorphic processor,” in IEEE Trans. Comput., vol. 53, no. 11, 2004, pp. 1363–1375.CrossRefGoogle Scholar
  9. 9.
    K. Compton and S. Hauck, “Reconfigurable computing: a survey of systems and software,” ACM Comput. Surv., vol. 34, no. 2, 2002, pp. 171–210.CrossRefGoogle Scholar
  10. 10.
    G. Berry and R. Sethi, “From regular expressions to deterministic automata,” Theor. Comput. Sci., vol. 48, no. 1, 1986, pp. 117–126.MATHCrossRefMathSciNetGoogle Scholar
  11. 11.
    J. E. Hopcroft and J. D. Ullman, “Introduction to Automata Theory, Languages and Computation, 2nd ed. Addison-Wesley, 2001.Google Scholar
  12. 12.
    R. W. Floyd and J. D. Ullman, “The compilation of regular expressions into integrated circuits,” J. Assoc. Comput. Mach., vol. 29, no. 3, 1982, pp. 603–622.MATHMathSciNetGoogle Scholar
  13. 13.
    A. Karlin, H. Trickey, and J. Ullman, “Experience with a regular expression compiler,” in Proc. of IEEE Conference on Computer Design/VLSI in Computers, 1983, pp. 656–665.Google Scholar
  14. 14.
    R. Sidhu and V. K. Prasanna, “Fast regular expression matching using FPGAs,” in Proc. of 9th IEEE Symposium on Field-Programmable Custom Computing Machines (FCCM’01), IEEE Computer Society Press, 2001, pp. 227–238.Google Scholar
  15. 15.
    A. Mukhopadhyay, “Hardware algorithms for non-numeric computation,” IEEE Trans. Comput., vol. C-28, no. 6, 1979, pp. 384–394.CrossRefGoogle Scholar
  16. 16.
    PCRE—Perl Compatible Regular Expressions, http://www.pcre.org/.
  17. 17.
    SNORT official web site, http://www.snort.org.
  18. 18.
    Bleeding Edge Threats web site, http://www.bleedingthreats.net.
  19. 19.
    I. Dubrawsky, “Firewall evolution—deep packet inspaction,” July 2003, http://www.securityfocus.com/infocus/1716.
  20. 20.
    M. Fisk and G. Varghese, “An analysis of fast string matching applied to content-based forwarding and intrusion detection,” in Techical Report CS2001-0670, University of California, San Diego, 2002.Google Scholar
  21. 21.
    B. L. Hutchings, R. Franklin, and D. Carver, “Assisting network intrusion detection with reconfigurable hardware,” in Proc. of 10th IEEE Symposium on Field-Programmable Custom Computing Machines (FCCM’02), IEEE Computer Society Press, 2002, pp. 111–120.Google Scholar
  22. 22.
    J. Moscola, J. Lockwood, R. P. Loui, and M. Pachos, “Implementation of a content-scanning module for an Internet firewall,” in Proc. of 11th IEEE Symposium on Field-Programmable Custom Computing Machines (FCCM’03), IEEE Computer Society Press, 2003, pp. 31–38.Google Scholar
  23. 23.
    C. R. Clark and D. E. Schimmel, “Scalable parallel pattern-matching on high-speed networks,” in Proc. of 12th IEEE Symposium on Field-Programmable Custom Computing Machines (FCCM’04), IEEE Computer Society Press, 2004, pp. 249–257.Google Scholar
  24. 24.
    I. Sourdis and D. Pnevmatikatos, “Pre-decoded CAMs for efficient and high-speed NIDS pattern matching,” in Proc. 12th IEEE Symposium on Field-Programmable Custom Computing Machines (FCCM’04), IEEE Computer Society Press, 2004, pp. 258–267.Google Scholar
  25. 25.
    I. Sourdis, D. Pnevmatikatos, S. Wong, and S. Vassiliadis, “A reconfigurable perfect-hashing scheme for packet inspection,” in Proc. of 15th Int’l Conference on Field Programmable Logic and Applications (FPL’05), Tampere, 2005, pp. 644–647.Google Scholar
  26. 26.
    G. Papadopoulos and D. Pnevmatikatos, “Hashing + Memory = Low Cost, exact pattern matching,” in Proc. 15th Int’l Conference on Field Programmable Logic and Applications (FPL’05), Tampere, 2005, pp. 39–44.Google Scholar
  27. 27.
    M. Roesch, “{S}nort—lightweight intrusion detection for networks,” in Proc. of 13th USENIX Conference on System Administration, Seattle, 1999, pp. 229–238.Google Scholar
  28. 28.
    M. Rabin and D. Scott, “Finite automata and their decision problems,” IBM J. Res. Develop., vol. 3, no. 2, 1959, pp. 114–125.MathSciNetCrossRefGoogle Scholar
  29. 29.
    R. McNaughton and H. Yamada, “Regular expressions and state graphs for automata,” IEEE Trans. Electron. Comput., vol. EC-9, no. 1, 1960, pp. 39–47.CrossRefGoogle Scholar
  30. 30.
    K. Thompson, “Regular expression search algorithm,” Commun. ACM, vol. 11, no. 6, 1968, pp. 419–422.MATHCrossRefGoogle Scholar
  31. 31.
    M. J. Foster, “Avoiding latch formation in regular expression recognizers,” IEEE Trans. Comput., vol. 38, no. 5, 1989, pp. 754–756.CrossRefMathSciNetGoogle Scholar
  32. 32.
    C. R. Clark and D. E. Schimmel, “Efficient reconfigurable logic circuit for matching complex network intrusion detection patterns,” in Proc. 13th Int’l Conference on Field Programmable Logic and Applications (FPL’03), Lisbon, 2003, pp. 956–959.Google Scholar
  33. 33.
    C.-H. Lin, C.-T. Huang, C.-P. Jiang, and S.-C. Chang, “Optimization of regular expression pattern matching circuits on FPGA,” in Proc. of Conference on Design, Automation and Test in Europe (DATE’06), Munich, 2006, pp. 12–17.Google Scholar
  34. 34.
    J. Moscola, Y. H. Cho, and J. W. Lockwood, “A scalable hybrid regular expression pattern matcher,” in Proc. of 14th IEEE Symposium on Field-Programmable Custom Computing Machines (FCCM’06), IEEE Computer Society Press, 2006, pp. 337–338.Google Scholar
  35. 35.
    Z. K. Baker and V. K. Prasanna, “A methodology for synthesis of efficient intrusion detection systems on FPGAs,” in Proc. 12th IEEE Symposium on Field-Programmable Custom Computing Machines (FCCM’04), IEEE Computer Society Press, 2004, pp. 135–144.Google Scholar
  36. 36.
    Z. K. Baker, H.-J. Jung, and V. K. Prasanna, “Regular expression software deceleration for intrusion detection systems,” in Proc. 16th Int’l Conference on Field Programmable Logic and Applications (FPL’06), Madrid, 2006, pp. 418–425.Google Scholar
  37. 37.
    B. C. Brodie, D. E. Taylor, and R. K. Cytron, “A scalable architecture for high-throughput regular-expression pattern matching,” Comput. Archit. News, vol. 34, no. 2, 2006, pp. 191–202 [also published in 33rd Int’l Symposium on Computer Architecture (ISCA’06)].Google Scholar
  38. 38.
    P. Sutton, “Partial character decoding for improved regular expression matching in FPGAs,” in Proc. of IEEE Int’l Conference on Field-Programmable Technology (FPT’04), Brisbane, 2004, pp. 25–32.Google Scholar
  39. 39.
    I. Sourdis, V. Dimopoulos, D. Pnevmatikatos, and S. Vassiliadis, “Packet pre-filtering for network intrusion detection,” in Proc. 2nd ACM/IEEE Symposium on Architectures for Networking and Communications Systems (ANCS’06), San Jose, 2006, pp. 183–192.Google Scholar
  40. 40.
    I. Sourdis and D. Pnevmatikatos, “Fast, large-scale string match for a 10 Gbps FPGA-based network intrusion detection system,” in Proc. of 13th Int’l Conference on Field Programmable Logic and Applications (FPL’03), Lisbon, 2003, pp. 880–889.Google Scholar
  41. 41.
    T. Sproull, G. Brebner, and C. Neely, “Mutable codesign for embedded protocol processing,” in Proc. of 15th Int’l Conference on Field Programmable Logic and Applications (FPL’05), Tampere, 2005, pp. 51–56.Google Scholar
  42. 42.
    J. C. Bispo, I. Sourdis, J. M.P. Cardoso, and S. Vassiliadis, “Regular expression matching for reconfigurable packet inspection,” in Proc. IEEE Int’l Conference on Field Programmable Technology (FPT’06), Bangkok, 2006, pp. 119–126.Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2007

Authors and Affiliations

  • Ioannis Sourdis
    • 1
  • João Bispo
    • 2
  • João M. P. Cardoso
    • 3
  • Stamatis Vassiliadis
    • 1
  1. 1.Computer EngineeringTU DelftDelftThe Netherlands
  2. 2.INESC-IDLisboaPortugal
  3. 3.Department of Informatics EngineeringIST/UTLLisboaPortugal

Personalised recommendations