Abstract
Estimating the number of substring matches is one of problems that estimate alphanumeric selectivity using statistical information for strings. In the context of alphanumeric selectivity estimation, a CS-tree (Count Suffix Tree), which is a variation of a suffix tree, has been used as a basic data structure to store statistical information for substrings. However, even though the CS-tree is useful to keep information about short strings such as name or title, the CS-tree has two drawbacks: one is that some count values that the CS-tree keeps can be incorrect, and the other is that it is almost impossible to build the CS-tree over long strings such as biological sequences.
Therefore, for estimating the number of substring matches in long strings, we propose a CQ-tree (Count Q-gram Tree), which keeps the exact count values of all substrings of length q or below q located in the long strings, and can be constructed in one scan of data strings.
Furthermore, on the basis of the CQ-tree, we return the lower and upper bounds that the number of occurrences of a query can reach to, together with the estimated count of the query pattern. These bounds are mathematically proved. To the best of our knowledge, our work is the first one that presents the lower and upper bounds among research activities about alphanumeric selectivity estimation.
This work was supported in part by the Brain Korea 21 Project and in part by the Ministry of Information & Communications, Korea, under the Information Technology Research Center (ITRC) Support Program in 2004.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Chen, Z., Jagadish, H.V., Korn, F., Koudas, N., Muthukrishnan, S., Ng, R.T., Srivastava, D.: Counting twig matches in a tree. In: ICDE, pp. 595–604 (2001)
Farach, M., Ferragina, P., Muthukrishnan, S.: Overcoming the memory bottlenect in suffix tree construction. In: 39th Annual Symposium on Foundations of Computer Science, pp. 174–185 (1998)
Fiala, E.R., Greene, D.H.: Data compression with finite windows. Comm. of the ACM 32, 490–505 (1989)
Grossi, R., Vitter, J.S.: Compressed suffix arrays and suffix trees with applications to text indexing and string matching. In: ACM Symposium on Theory of Computing, pp. 397–456 (2000)
Horowitz, E., Sahni, S., Mehta, D.: Fundamentals of data structures in C++. W H Freeman & Co., New York (1995)
Jagadish, H.V., Kapitskaia, O., Ng, R.T., Srivastava, D.: Multi-dimensional substring selectivity estimation. In: VLDB, pp. 387–398 (1999)
Jagadish, H.V., Ng, R.T., Srivastava, D.: Substring selectivity estimation. In: PODS, pp. 249–260 (1999)
Jagadish, H.V., Ng, R.T., Srivastava, D.: On effective multi-dimensional indexing for strings. In: ACM SIGMOD, pp. 403–414 (2000)
Krishnan, P., Vitter, J.S., Iyer, B.: Estimating alphanumeric selectivity in the presence of wildcards. In: ACM SIGMOD, pp. 282–293 (1996)
McCreight, E.M.: A space-economical suffix tree construction algorithm. Journal of the ACM 23, 262–272 (1976)
Navarro, G., Baeza-Yates, R.: A practical q-gram index for text retrieval allowing errors. CLEI Electronic Journal 1 (1998)
Navarro, G., Sutinen, E., Tanninen, J., Tarhio, J.: Indexing text with approximate q-grams. In: Combinatorial Pattern Matching, pp. 350–363 (2000)
NCBI (2001), http://ncbi.nlm.nih.gov
Weiner, P.: Linear pattern matching algorithms. In: IEEE 14th Annual Symp. On Switching and Automata Theory, pp. 1–11 (1990)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Bae, J., Lee, S. (2005). Estimating the Number of Substring Matches in Long String Databases. In: Zhang, Y., Tanaka, K., Yu, J.X., Wang, S., Li, M. (eds) Web Technologies Research and Development - APWeb 2005. APWeb 2005. Lecture Notes in Computer Science, vol 3399. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-31849-1_15
Download citation
DOI: https://doi.org/10.1007/978-3-540-31849-1_15
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-25207-8
Online ISBN: 978-3-540-31849-1
eBook Packages: Computer ScienceComputer Science (R0)