Advertisement

The VLDB Journal

, Volume 12, Issue 2, pp 102–119 | Cite as

RE-tree: an efficient index structure for regular expressions

  • Chee-Yong ChanEmail author
  • Minos Garofalakis
  • Rajeev Rastogi
Original Paper

Abstract.

Due to their expressive power, regular expressions (REs) are quickly becoming an integral part of language specifications for several important application scenarios. Many of these applications have to manage huge databases of RE specifications and need to provide an effective matching mechanism that, given an input string, quickly identifies the REs in the database that match it. In this paper, we propose the RE-tree, a novel index structure for large databases of RE specifications. Given an input query string, the RE-tree speeds up the retrieval of matching REs by focusing the search and comparing the input string with only a small fraction of REs in the database. Even though the RE-tree is similar in spirit to other tree-based structures that have been proposed for indexing multidimensional data, RE indexing is significantly more challenging since REs typically represent infinite sets of strings with no well-defined notion of spatial locality. To address these new challenges, our RE-tree index structure relies on novel measures for comparing the relative sizes of infinite regular languages. We also propose innovative solutions for the various RE-tree operations including the effective splitting of RE-tree nodes and computing a "tight" bounding RE for a collection of REs. Finally, we demonstrate how sampling-based approximation algorithms can be used to significantly speed up the performance of RE-tree operations. Preliminary experimental results with moderately large synthetic data sets indicate that the RE-tree is effective in pruning the search space and easily outperforms naive sequential search approaches.

Keywords:

Regular expressions Index structure Size measures Sampling-based approximations 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1. Altinel M, Franklin MJ (2000) Efficient filtering of XML documents for selective dissemination of information. In: Proceedings of VLDB, Cairo, September 2000, pp 53--64Google Scholar
  2. 2. Baeza-Yates R, Gonnet G (1996) Fast text searching for regular expressions or automaton searching on a trie. J ACM 43(6):915--936Google Scholar
  3. 3. Baeza-Yates R, Ribeiro-Neto B, Navarro G (1999) Indexing and searching. In: Modern information retrieval. ACM Press, New York, pp 191--228Google Scholar
  4. 4. Beckmann N, Kriegel H-P, Schneider R, Seeger B (1990) The R*-Tree: an efficient and robust access method for points and rectangles. In: Proceedings of SIGMOD, Atlantic City, NJ, May 1990, pp 322--331Google Scholar
  5. 5. Chan C-Y, Felber P, Garofalakis M, Rastogi R (2002) Efficient filtering of XML documents with XPath expressions. In: Proceedings of the 18th international conference on data engineering, San Jose, February 2002, pp 235--244Google Scholar
  6. 6. Chan CY, Garofalakis M, Rastogi R (2002) RE-Tree: an efficient index structure for regular expressions. In: Proceedings of VLDB, Hong Kong, August 2002, pp 251--262Google Scholar
  7. 7. Clark J, DeRose S (1999) XML path language (XPath) v. 1.0. W3C Recommendation, http://www.w3.org./TR/xpathGoogle Scholar
  8. 8. Cochran WG (1977) Sampling techniques. Wiley, New YorkGoogle Scholar
  9. 9. Intel Corporation (2000) Intel NetStructure XML accelerators, http://www.intel.com/netstructure/products/xml/accelerators.htmGoogle Scholar
  10. 10. Deutsch A, Fernandez M, Suciu D (1999) Storing semistructured data with STORED. In: Proceedings of the ACM SIGMOD conference on management of data, Philadelphia, June 1999, pp 431--442Google Scholar
  11. 11. Diao Y, Fischer P, Franklin MJ, To R (2002) YFilter: efficient and scalable filtering of XML documents. In: Proceedings of the 18th international conference on data engineering, San Jose, February 2002, pp 341--342Google Scholar
  12. 12. Garey MR, Johnson DS (1979) Computers and intractability: a guide to the theory of NP-Completeness. Freeman, San FranciscoGoogle Scholar
  13. 13. Garofalakis M, Gionis A, Rastogi R, Seshadri S, Shim K (2000) XTRACT: a system for extracting document type descriptors from XML documents. In: Proceedings of SIGMOD, Dallas, May 2000, pp 165--176Google Scholar
  14. 14. Guttman A (1984) R-Trees: a dynamic index structure for spatial searching. In: Proceedings of SIGMOD, Boston, June 1984, pp 47--57Google Scholar
  15. 15. Hopcroft JE, Ullman JD (1979) Introduction to automata theory, languages, and computation. Addison-Wesley, Reading, MAGoogle Scholar
  16. 16. Ishikawa Y, Kitagawa H, Ohbo N (1993) Evaluation of signature files as set access facilities in OODBs. In: Proceedings of SIGMOD, Washington, DC, May 1993, pp 247--256Google Scholar
  17. 17. Kannan S, Sweedyk Z, Mahaney S (1995) Counting and random generation of strings in regular languages. In: Proceedings of SODA, San Francisco, January 1995, pp 551--557Google Scholar
  18. 18. Kilpelainen P, Mannila H, Ukkonen E (1995) MDL learning of unions of simple pattern languages from positive examples. In: Proceedings of EuroCOLT, Barcelona, March 1995, pp 252--260Google Scholar
  19. 19. Lakshmanan LVS, Parthasarathy S (2002) On efficient matching of streaming XML documents and queries. In: Proceedings of EDBT, Prague, March 2002, pp 142--160Google Scholar
  20. 20. Quinlan JR, Rivest RL (1989) Inferring decision trees using the minimum description length principle. Information and Computation 80:227--248Google Scholar
  21. 21. Rissanen J (1978) Modeling by shortest data description. Automatica 14:465--471Google Scholar
  22. 22. Stewart JW (1998) BGP4 inter-domain routing in the Internet. Addison-Wesley, Reading, MAGoogle Scholar
  23. 23. Wu S, Manber U (1992) Fast text searching allowing errors. Commun ACM 35(10):83--91Google Scholar

Copyright information

© Springer-Verlag Berlin/Heidelberg 2003

Authors and Affiliations

  • Chee-Yong Chan
    • 1
    Email author
  • Minos Garofalakis
    • 1
  • Rajeev Rastogi
    • 1
  1. 1.Bell LaboratoriesLucent TechnologiesMurray HillUSA

Personalised recommendations