Pattern Analysis and Applications

, Volume 9, Issue 2–3, pp 177–187 | Cite as

A novel look-ahead optimization strategy for trie-based approximate string matching

Theoretical Advances
First Online:
Received:
Accepted:

Abstract

This paper deals with the problem of estimating a transmitted string X* by processing the corresponding string Y, which is a noisy version of X*. We assume that Y contains substitution, insertion, and deletion errors, and that X* is an element of a finite (but possibly, large) dictionary, H. The best estimate X+ of X*, is defined as that element of H which minimizes the generalized Levenshtein distance D(X, Y) between X and Y such that the total number of errors is not more than K, for all X ∈H. The trie is a data structure that offers search costs that are independent of the document size. Tries also combine prefixes together, and so by using tries in approximate string matching we can utilize the information obtained in the process of evaluating any one D(Xi, Y), to compute any other D(Xj, Y), where Xi and Xj share a common prefix. In the artificial intelligence (AI) domain, branch and bound (BB) schemes are used when we want to prune paths that have costs above a certain threshold. These techniques have been applied to prune, for example, game trees. In this paper, we present a new BB pruning strategy that can be applied to dictionary-based approximate string matching when the dictionary is stored as a trie. The new strategy attempts to look ahead at each node, c, before moving further, by merely evaluating a certain local criterion at c. The search algorithm according to this pruning strategy will not traverse inside the subtrie(c) unless there is a “hope” of determining a suitable string in it. In other words, as opposed to the reported trie-based methods (Kashyap and Oommen in Inf Sci 23(2):123–142, 1981; Shang and Merrettal in IEEE Trans Knowledge Data Eng 8(4):540–547, 1996), the pruning is done a priori before even embarking on the edit distance computations. The new strategy depends highly on the variance of the lengths of the strings in H. It combines the advantages of partitioning the dictionary according to the string lengths, and the advantages gleaned by representing H using the trie data structure. The results demonstrate a marked improvement (up to 30% when costs are of a 0/1 form, and up to 47% when costs are general) with respect to the number of operations needed on three benchmark dictionaries.

Keywords

Trie-based syntactic pattern recognition Approximate string matching Noisy syntactic recognition using tries Branch and bound techniques Pruning 

References

  1. 1.
    Altschul SF, Gish W, Miller W, Myers EW, Lipman DJ (1990) A basic local alignment search tool. J Mol Biol 215:403–410CrossRefGoogle Scholar
  2. 2.
    Baeza-Yates RA, Gonnet GH (1982) A new approach to text searching. In: Annual ACM-SIGIR conference on information retrieval, Cambridge, MA, June 1982, pp 168–175Google Scholar
  3. 3.
    Bentley J, Sedgewick R (1997) Fast algorithms for sorting and searching strings. In: Eighth annual ACM-SIAM symposium on discrete algorithms, New Orleans, January 1997, pp 360–369Google Scholar
  4. 4.
    Bucher P, Hoffmann K (1996) A sequence similarity search algorithm based on a probabilistic interpretation of an alignment scoring system. In: Proceedings of the fourth international conference on intelligent systems for molecular biology, ISMB, vol 96, pp 44–51Google Scholar
  5. 5.
    Bunke H (1993) Structural and syntactic pattern recognition. In: Chen CH, Pau LF, Wang PSP (eds) Handbook of pattern recognition and computer vision. World Scientific, SingaporeGoogle Scholar
  6. 6.
    Bunke H, Csirik J (1993) Parametric string edit distance and its application to pattern recognition. IEEE Trans Syst Man Cybern SMC-25(1):202–206Google Scholar
  7. 7.
    Chang W, Lawler E (1992) Approximate string matching in sublinear expected time. In: 13th annual symposium on foundations of computer science, St.~Louis, Missouri, October 1992. IEEE Computer Society Press, pp 116–124Google Scholar
  8. 8.
    Clement J, Flajolet P, Vallee B (1998) The analysis of hybrid trie structures. In: Proceedings of the annual ACM–SIAM symposium on discrete algorithms, San Francisco, CA, pp 531–539Google Scholar
  9. 9.
    Crochemore M, Landau GM, Ziv-Ukleson M (1973) A subquadratic sequence alignment algorithm for unrestricted scoring matrices. SIAM J 32(6):1654–1673CrossRefGoogle Scholar
  10. 11.
    Dewey G (1923) Relative frequency of English speech sounds. Harvard University Press, Cambridge, MAGoogle Scholar
  11. 11.
    Du M, Chang S (1994) An approach to designing very fast approximate string matching algorithms. IEEE Trans Knowledge Data Eng 6(4):620–633CrossRefGoogle Scholar
  12. 12.
    Firebaugh M (1988) Artificial intelligence: a knowledge-based approach. Boyd and Fraser, BostonGoogle Scholar
  13. 13.
    Hirschberg DS (1975) A linear space algorithm for computing maximal common subsequence. Commun ACM 18(6):341–343MATHMathSciNetCrossRefGoogle Scholar
  14. 14.
    Hunt JW, Szymanski TG (1977) A fast algorithm for computing longest common subsequences. Commun Assoc Comput Mach 20:350–353MATHMathSciNetGoogle Scholar
  15. 15.
    Kashyap RL, Oommen BJ (1981) An effective algorithm for string correction using generalized edit distances -I: description of the algorithm and its optimality. Inf Sci 23(2):123–142CrossRefGoogle Scholar
  16. 16.
    Levenshtein A (1966) Binary codes capable of correcting deletions, insertions and reversals. Sov Phys Dokl 10:707–710MathSciNetGoogle Scholar
  17. 17.
    Masek WJ, Paterson MS (1980) A faster algorithm computing string edit distances. J Comput Syst Sci 20:18–31MATHMathSciNetCrossRefGoogle Scholar
  18. 18.
    Navarro G (2001) A guided tour to approximate string matching. ACM Comput Surv 33(1):31–88CrossRefGoogle Scholar
  19. 19.
    Oflazer K (1996) Error-tolerant finite state recognition with applications to morphological analysis and spelling correction. Comput Linguist 22(1):73–89Google Scholar
  20. 20.
    Oommen BJ (1987) Recognition of noisy subsequences using constrained edit distances. IEEE Trans Pattern Anal Mach Intel PAMI 9:676–685MATHCrossRefGoogle Scholar
  21. 21.
    Oommen BJ, Badr G (2004) Dictionary-based syntactic pattern recognition using tries. In: Proceedings of the joint IARR international workshops SSPR 2004 and SPR 2004, Libon, August 2004Google Scholar
  22. 22.
    Oommen BJ, Kashyap RL (1998) A formal theory for optimal and information theoretic syntactic pattern recognition. Pattern Recognit 31:1159–1177CrossRefGoogle Scholar
  23. 23.
    Oommen BJ, Loke RKS (1999) Designing syntactic pattern classifiers using vector quantization and parametric string editing. IEEE Trans Syst Man Cybern SMC-29:881-888Google Scholar
  24. 24.
    Oommen BJ, Loke RKS (2006) Syntactic pattern recognition involving traditional and generalized transposition errors: attaining the information theoretic bound (submitted)Google Scholar
  25. 25.
    Peterson JL (1980) Computer programs for detecting and correcting spelling errors. Commun Assoc Comput Mach 23:676–687Google Scholar
  26. 26.
    Sankoff D, Kruskal JB (1983) Time warps, string edits and macromolecules: the theory and practice of sequence comparison. Addison–Wesley, Reading, MAGoogle Scholar
  27. 27.
    Shang H, Merrettal T (1996) Tries for approximate string matching. IEEE Trans Knowledge Data Eng 8(4):540–547CrossRefGoogle Scholar
  28. 28.
    Stephen GA (2000) String searching algorithms, Lecture notes series on computing, vol 6, World Scientific, Sihgapore, NJGoogle Scholar
  29. 29.
    Ukkonen E (1985) Algorithm for approximate string matching. Inf control 64:100–118MATHMathSciNetCrossRefGoogle Scholar
  30. 30.
    Wagner RA (1974) Order-n correction for regular languages. Commun ACM 17:265–268MATHCrossRefGoogle Scholar
  31. 31.
    Wagner R, Fischer A (1974) The string-to-string correction problem. J Assoc Comput Machinery (ACM) 21:168–173MATHMathSciNetGoogle Scholar
  32. 32.
    Wu S, Manber U (1992) Fast text searching allowing errors. Commmun ACM 35(10):83–91CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London Limited 2006

Authors and Affiliations

  1. 1.School of Computer ScienceCarleton UniversityOttawaCanada

Personalised recommendations