Fast Relative Lempel-Ziv Self-index for Similar Sequences

  • Huy Hoang Do
  • Jesper Jansson
  • Kunihiko Sadakane
  • Wing-Kin Sung
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7285)

Abstract

Recent advances in biotechnology and web technology are generating huge collections of similar strings. People now face the problem of storing them compactly while supporting fast pattern searching. One compression scheme called relative Lempel-Ziv compression uses textual substitutions from a reference text as follows: Given a (large) set S of strings, represent each string in S as a concatenation of substrings from a reference string R. This basic scheme gives a good compression ratio when every string in S is similar to R, but does not provide any pattern searching functionality. Here, we describe a new data structure that supports fast pattern searching.

Keywords

Data Structure Range Query Query Time Compression Scheme Query Pattern 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    The 1000 Genomes Project Consortium: A map of human genome variation from population-scale sequencing. Nature 467(7319), 1061–1073 (2010)Google Scholar
  2. 2.
    Bille, P., Landau, G.M., Raman, R., Sadakane, K., Satti, S.R., Weimann, O.: Random access to grammar-compressed strings. In: SODA, pp. 373–389 (2011)Google Scholar
  3. 3.
    Cao, M.D., Dix, T.I., Allison, L., Mears, C.: A simple statistical algorithm for biological sequence compression. In: DCC, pp. 43–52 (2007)Google Scholar
  4. 4.
    Chan, T.M., Larsen, K.G., Pătraşcu, M.: Orthogonal range searching on the RAM, revisited. In: SoCG, pp. 1–10 (2011)Google Scholar
  5. 5.
    Christley, S., Lu, Y., Li, C., Xie, X.: Human genomes as email attachments. Bioinformatics 25(2), 274–275 (2009)CrossRefGoogle Scholar
  6. 6.
    Claude, F., Navarro, G.: Self-indexed Text Compression Using Straight-Line Programs. In: Královič, R., Niwiński, D. (eds.) MFCS 2009. LNCS, vol. 5734, pp. 235–246. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  7. 7.
    Ferragina, P., Manzini, G.: Compression boosting in optimal linear time using the Burrows-Wheeler Transform. In: SODA, pp. 655–663 (2004)Google Scholar
  8. 8.
    Ferragina, P., Manzini, G.: Indexing compressed text. Journal of the ACM 52(4), 552–581 (2005)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Fischer, J., Heun, V.: A New Succinct Representation of RMQ-Information and Improvements in the Enhanced Suffix Array. In: Chen, B., Paterson, M., Zhang, G. (eds.) ESCAPE 2007. LNCS, vol. 4614, pp. 459–470. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  10. 10.
    Gagie, T., Gawrychowski, P., Kärkkäinen, J., Nekrich, Y., Puglisi, S.J.: A Faster Grammar-Based Self-index. In: Dediu, A.-H., Martín-Vide, C. (eds.) LATA 2012. LNCS, vol. 7183, pp. 240–251. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  11. 11.
    Golynski, A., Munro, J.I., Rao, S.S.: Rank/select operations on large alphabets: a tool for text indexing. In: SODA, pp. 368–373 (2006)Google Scholar
  12. 12.
    Grumbach, S., Tahi, F.: Compression of DNA sequences. In: DCC, pp. 340–350 (1993)Google Scholar
  13. 13.
    Huang, S., Lam, T.W., Sung, W.K., Tam, S.L., Yiu, S.M.: Indexing Similar DNA Sequences. In: Chen, B. (ed.) AAIM 2010. LNCS, vol. 6124, pp. 180–190. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  14. 14.
    Kreft, S., Navarro, G.: LZ77-like compression with fast random access. In: DCC, pp. 239–248 (2010)Google Scholar
  15. 15.
    Kreft, S., Navarro, G.: Self-indexing Based on LZ77. In: Giancarlo, R., Manzini, G. (eds.) CPM 2011. LNCS, vol. 6661, pp. 41–54. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  16. 16.
    Kuruppu, S., Puglisi, S.J., Zobel, J.: Relative Lempel-Ziv Compression of Genomes for Large-Scale Storage and Retrieval. In: Chavez, E., Lonardi, S. (eds.) SPIRE 2010. LNCS, vol. 6393, pp. 201–206. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  17. 17.
    Kuruppu, S., Puglisi, S.J., Zobel, J.: Reference Sequence Construction for Relative Compression of Genomes. In: Grossi, R., Sebastiani, F., Silvestri, F. (eds.) SPIRE 2011. LNCS, vol. 7024, pp. 420–425. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  18. 18.
    Mäkinen, V., Navarro, G.: Implicit Compression Boosting with Applications to Self-indexing. In: Ziviani, N., Baeza-Yates, R. (eds.) SPIRE 2007. LNCS, vol. 4726, pp. 229–241. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  19. 19.
    Mäkinen, V., Navarro, G., Sirén, J., Välimäki, N.: Storage and retrieval of highly repetitive sequence collections. J. of Computational Biology 17(3), 281–308 (2010)CrossRefGoogle Scholar
  20. 20.
    Muthukrishnan, S.: Efficient algorithms for document retrieval problems. In: SODA, pp. 657–666 (2002)Google Scholar
  21. 21.
    Nekrich, Y.: Orthogonal range searching in linear and almost-linear space. Computational Geometry 42(4), 342–351 (2009)MathSciNetMATHCrossRefGoogle Scholar
  22. 22.
    Pătraşcu, M.: Succincter. In: FOCS, pp. 305–313 (2008)Google Scholar
  23. 23.
    Rytter, W.: Application of Lempel-Ziv factorization to the approximation of grammar-based compression. Theoretical Computer Science 302, 211–222 (2003)MathSciNetMATHCrossRefGoogle Scholar
  24. 24.
    Sirén, J., Välimäki, N., Mäkinen, V., Navarro, G.: Run-Length Compressed Indexes Are Superior for Highly Repetitive Sequence Collections. In: Amir, A., Turpin, A., Moffat, A. (eds.) SPIRE 2008. LNCS, vol. 5280, pp. 164–175. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  25. 25.
    Willard, D.E.: Log-logarithmic worst-case range queries are possible in space Θ(N). Information Processing Letters 17(2), 81–84 (1983)MathSciNetMATHCrossRefGoogle Scholar
  26. 26.
    Ziv, J., Lempel, A.: A universal algorithm for sequential data compression. IEEE Transactions on Information Theory 23(3), 337–343 (1977)MathSciNetMATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Huy Hoang Do
    • 1
  • Jesper Jansson
    • 2
  • Kunihiko Sadakane
    • 3
  • Wing-Kin Sung
    • 1
  1. 1.National University of Singapore, COM 1Singapore
  2. 2.Ochanomizu UniversityTokyoJapan
  3. 3.National Institute of InformaticsTokyoJapan

Personalised recommendations