Super-Linear Indices for Approximate Dictionary Searching
We present experimental analysis of approximate search algorithms that involve indexing of deletion neighborhoods. These methods require huge indices whose sizes grow exponentially with respect to the maximum allowable number of errors k. Despite extraordinary space requirements, the super-linear indices are of great interest, because they provide some of the shortest retrieval times.
A straightforward implementation that creates a hash index directly over residual strings (obtained by deletions from dictionary words) is not space efficient. Rather than memorizing complete residual strings, we record only deleted characters and their respective positions. These data are indexed using a perfect hash function computed for a set of residual dictionary strings .
We carry out an experimental evaluation of this approach against several well-known benchmarks (including FastSS, which stores residual strings directly ). Experiments show that our implementation has a comparable or superior performance to that of the fastest benchmarks. At the same time, our implementation requires 4-8 times less space as compared to FastSS.
Keywordswildcard neighborhood generation reduced alphabet neighborhood generation Mor-Fraenkel method perfect hashing FastSS
Unable to display preview. Download preview PDF.
- 1.Behm, A., Vernica, R., Alsubaiee, S., Ji, S., Lu, J., Jin, L., Lu, Y., Li, C.: UCI Flamingo Package 4.0 (2010)Google Scholar
- 3.Bocek, T., Hunt, E., Stiller, B.: Fast similarity search in large dictionaries, Technical report No. ifi-2007.02, Department of Informatics (IFI), University of Zurich (2007)Google Scholar
- 4.Botelho, F.C.: Near-Optimal Space Perfect Hashing Algorithms. PhD thesis, Graduate Program in Computer Science, Federal University of Minas Gerais, Brazil (2008)Google Scholar
- 5.Boytsov, L.: Indexing methods for approximate dictionary searching: Comparative analysis. J. Exp. Algorithmics 16, 1.1:1.1–1.1:1.91 (2011)Google Scholar
- 7.Cole, R., Gottlieb, L.A., Lewenstein, M.: Dictionary matching and indexing with errors and don’t cares. In: STOC 2004: Proceedings of the Thirty-Sixth Annual ACM Symposium on Theory of Computing, pp. 91–100. ACM (2004)Google Scholar
- 10.Gorin, R.E.: SPELL: Spelling check and correction program, Online documentation: Describes operation of PDP-10 SPELL program (1971), http://pdp-10.trailing-edge.com/decuslib10-03/01/43,50270/spell.doc.html (accessed May 28, 2012)
- 12.Karch, D., Luxen, D., Sanders, P.: Improved fast similarity search in dictionaries. CoRR abs/1008.1191 (2010)Google Scholar
- 13.Knuth, D.: The Art of Computer Programming. Sorting and Searching., 1st edn., vol. 3. Addison-Wesley (1973)Google Scholar