Abstract
In this paper we describe a regular expression pattern matching approach for reconfigurable hardware. Following a Non-deterministic Finite Automata direction, we introduce three new basic building blocks to support constraint repetitions syntaxes more efficiently than previous works. In addition, a number of optimization techniques are employed to reduce the area cost of the designs and maximize performance. Our design methodology is supported by a tool that automatically generates the circuitry for the given regular expressions and outputs Hardware Description Language representations ready for logic synthesis. The proposed approach is evaluated on network Intrusion Detection Systems (IDS). Recent IDS use regular expressions to represent hazardous packet payload contents. They require high-speed packet processing providing a challenging case study for pattern matching using regular expressions. We use a number of IDS rulesets to show that our approach scales well as the number of regular expressions increases, and present a step-by-step optimization to survey the benefits of our techniques. The synthesis tool described in this study is used to generate hardware engines to match 300 to 1,500 IDS regular expressions using only 10–45 K logic cells and achieving throughput of 1.6–2.2 and 2.4–3.2 Gbps on Virtex2 and Virtex4 devices, respectively. Concerning the throughput per area required per matching non-Meta character, our hardware engines are 10–20× more efficient than previous Field Programmable Gate Array approaches. Furthermore, the generated designs have comparable area requirements to current application-specific integrated circuit solutions.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
S. Stephens, J. Y. Chen, M. G. Davidson, S. Thomas, and B. M. Trute, “Oracle database 10 g: a platform for blast search and regular expression pattern matching in life sciences,” Nucleic Acids Res., vol. 33 (database-Issue), 2005, pp. 675–679.
S. Ray and M. Craven, “Learning statistical models for annotating proteins with function information using biomedical text,” BMC Bioinformatics, vol. 6, Suppl. 1, 2005, p. S:18.
J.-M. Champarnaud, F. Coulon, and T. Paranthoen, “Compact and fast algorithms for safe regular expression search,” Int. J. Comput. Math., vol. 81, no. 4, 2004, pp. 383–401.
F. Yu, Z. Chen, Y. Diao, T. V. Lakshman, and R. H. Katz, “Fast and memory-efficient regular expression matching for deep packet inspection,” in Proc. 2nd ACM/IEEE Symposium on Architectures for Networking and Communications Systems (ANCS’06), ACM Press, 2006, pp. 93–102.
S. Kumar, S. Dharmapurikar, F. Yu, P. Crowley, and J. Turner, “Algorithms to accelerate multiple regular expressions matching for deep packet inspection,” Comput. Commun. Rev., vol. 36, no. 4, 2006, pp. 339–350.
F. Yu, Z. Chen, Y. Diao, T. Lakshman, and R. H. Katz, “Fast and memory-efficient regular expression matching for deep packet inspection,” EECS Department, University of California, Berkeley, Tech. Rep. UCB/EECS-2006-76, May 22 2006. [Online]. Available: http://www.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-2006-76.html.
S. Kumar, J. Turner, and J. Williams, “Advanced algorithms for fast and scalable deep packet inspection,” in Proc. of ACM/IEEE Symposium on Architecture for Networking and Sommunications Systems (ANCS’06), New York, ACM Press, 2006, pp. 81–92.
S. Vassiliadis, S. Wong, G. N. Gaydadjiev, K. Bertels, G. Kuzmanov, and E. M. Panainte, “The Molen polymorphic processor,” in IEEE Trans. Comput., vol. 53, no. 11, 2004, pp. 1363–1375.
K. Compton and S. Hauck, “Reconfigurable computing: a survey of systems and software,” ACM Comput. Surv., vol. 34, no. 2, 2002, pp. 171–210.
G. Berry and R. Sethi, “From regular expressions to deterministic automata,” Theor. Comput. Sci., vol. 48, no. 1, 1986, pp. 117–126.
J. E. Hopcroft and J. D. Ullman, “Introduction to Automata Theory, Languages and Computation, 2nd ed. Addison-Wesley, 2001.
R. W. Floyd and J. D. Ullman, “The compilation of regular expressions into integrated circuits,” J. Assoc. Comput. Mach., vol. 29, no. 3, 1982, pp. 603–622.
A. Karlin, H. Trickey, and J. Ullman, “Experience with a regular expression compiler,” in Proc. of IEEE Conference on Computer Design/VLSI in Computers, 1983, pp. 656–665.
R. Sidhu and V. K. Prasanna, “Fast regular expression matching using FPGAs,” in Proc. of 9th IEEE Symposium on Field-Programmable Custom Computing Machines (FCCM’01), IEEE Computer Society Press, 2001, pp. 227–238.
A. Mukhopadhyay, “Hardware algorithms for non-numeric computation,” IEEE Trans. Comput., vol. C-28, no. 6, 1979, pp. 384–394.
PCRE—Perl Compatible Regular Expressions, http://www.pcre.org/.
SNORT official web site, http://www.snort.org.
Bleeding Edge Threats web site, http://www.bleedingthreats.net.
I. Dubrawsky, “Firewall evolution—deep packet inspaction,” July 2003, http://www.securityfocus.com/infocus/1716.
M. Fisk and G. Varghese, “An analysis of fast string matching applied to content-based forwarding and intrusion detection,” in Techical Report CS2001-0670, University of California, San Diego, 2002.
B. L. Hutchings, R. Franklin, and D. Carver, “Assisting network intrusion detection with reconfigurable hardware,” in Proc. of 10th IEEE Symposium on Field-Programmable Custom Computing Machines (FCCM’02), IEEE Computer Society Press, 2002, pp. 111–120.
J. Moscola, J. Lockwood, R. P. Loui, and M. Pachos, “Implementation of a content-scanning module for an Internet firewall,” in Proc. of 11th IEEE Symposium on Field-Programmable Custom Computing Machines (FCCM’03), IEEE Computer Society Press, 2003, pp. 31–38.
C. R. Clark and D. E. Schimmel, “Scalable parallel pattern-matching on high-speed networks,” in Proc. of 12th IEEE Symposium on Field-Programmable Custom Computing Machines (FCCM’04), IEEE Computer Society Press, 2004, pp. 249–257.
I. Sourdis and D. Pnevmatikatos, “Pre-decoded CAMs for efficient and high-speed NIDS pattern matching,” in Proc. 12th IEEE Symposium on Field-Programmable Custom Computing Machines (FCCM’04), IEEE Computer Society Press, 2004, pp. 258–267.
I. Sourdis, D. Pnevmatikatos, S. Wong, and S. Vassiliadis, “A reconfigurable perfect-hashing scheme for packet inspection,” in Proc. of 15th Int’l Conference on Field Programmable Logic and Applications (FPL’05), Tampere, 2005, pp. 644–647.
G. Papadopoulos and D. Pnevmatikatos, “Hashing + Memory = Low Cost, exact pattern matching,” in Proc. 15th Int’l Conference on Field Programmable Logic and Applications (FPL’05), Tampere, 2005, pp. 39–44.
M. Roesch, “{S}nort—lightweight intrusion detection for networks,” in Proc. of 13th USENIX Conference on System Administration, Seattle, 1999, pp. 229–238.
M. Rabin and D. Scott, “Finite automata and their decision problems,” IBM J. Res. Develop., vol. 3, no. 2, 1959, pp. 114–125.
R. McNaughton and H. Yamada, “Regular expressions and state graphs for automata,” IEEE Trans. Electron. Comput., vol. EC-9, no. 1, 1960, pp. 39–47.
K. Thompson, “Regular expression search algorithm,” Commun. ACM, vol. 11, no. 6, 1968, pp. 419–422.
M. J. Foster, “Avoiding latch formation in regular expression recognizers,” IEEE Trans. Comput., vol. 38, no. 5, 1989, pp. 754–756.
C. R. Clark and D. E. Schimmel, “Efficient reconfigurable logic circuit for matching complex network intrusion detection patterns,” in Proc. 13th Int’l Conference on Field Programmable Logic and Applications (FPL’03), Lisbon, 2003, pp. 956–959.
C.-H. Lin, C.-T. Huang, C.-P. Jiang, and S.-C. Chang, “Optimization of regular expression pattern matching circuits on FPGA,” in Proc. of Conference on Design, Automation and Test in Europe (DATE’06), Munich, 2006, pp. 12–17.
J. Moscola, Y. H. Cho, and J. W. Lockwood, “A scalable hybrid regular expression pattern matcher,” in Proc. of 14th IEEE Symposium on Field-Programmable Custom Computing Machines (FCCM’06), IEEE Computer Society Press, 2006, pp. 337–338.
Z. K. Baker and V. K. Prasanna, “A methodology for synthesis of efficient intrusion detection systems on FPGAs,” in Proc. 12th IEEE Symposium on Field-Programmable Custom Computing Machines (FCCM’04), IEEE Computer Society Press, 2004, pp. 135–144.
Z. K. Baker, H.-J. Jung, and V. K. Prasanna, “Regular expression software deceleration for intrusion detection systems,” in Proc. 16th Int’l Conference on Field Programmable Logic and Applications (FPL’06), Madrid, 2006, pp. 418–425.
B. C. Brodie, D. E. Taylor, and R. K. Cytron, “A scalable architecture for high-throughput regular-expression pattern matching,” Comput. Archit. News, vol. 34, no. 2, 2006, pp. 191–202 [also published in 33rd Int’l Symposium on Computer Architecture (ISCA’06)].
P. Sutton, “Partial character decoding for improved regular expression matching in FPGAs,” in Proc. of IEEE Int’l Conference on Field-Programmable Technology (FPT’04), Brisbane, 2004, pp. 25–32.
I. Sourdis, V. Dimopoulos, D. Pnevmatikatos, and S. Vassiliadis, “Packet pre-filtering for network intrusion detection,” in Proc. 2nd ACM/IEEE Symposium on Architectures for Networking and Communications Systems (ANCS’06), San Jose, 2006, pp. 183–192.
I. Sourdis and D. Pnevmatikatos, “Fast, large-scale string match for a 10 Gbps FPGA-based network intrusion detection system,” in Proc. of 13th Int’l Conference on Field Programmable Logic and Applications (FPL’03), Lisbon, 2003, pp. 880–889.
T. Sproull, G. Brebner, and C. Neely, “Mutable codesign for embedded protocol processing,” in Proc. of 15th Int’l Conference on Field Programmable Logic and Applications (FPL’05), Tampere, 2005, pp. 51–56.
J. C. Bispo, I. Sourdis, J. M.P. Cardoso, and S. Vassiliadis, “Regular expression matching for reconfigurable packet inspection,” in Proc. IEEE Int’l Conference on Field Programmable Technology (FPT’06), Bangkok, 2006, pp. 119–126.
Author information
Authors and Affiliations
Corresponding author
Additional information
This work was supported in part by the European Commission in the context of the Scalable computer Architectures (SARC) integrated project #27648 (FP6).
Rights and permissions
Open Access This is an open access article distributed under the terms of the Creative Commons Attribution Noncommercial License ( https://creativecommons.org/licenses/by-nc/2.0 ), which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
About this article
Cite this article
Sourdis, I., Bispo, J., Cardoso, J.M.P. et al. Regular Expression Matching in Reconfigurable Hardware. J Sign Process Syst Sign Image 51, 99–121 (2008). https://doi.org/10.1007/s11265-007-0131-0
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11265-007-0131-0