Skip to main content
SpringerLink
Log in

Towards an Optimal Space-and-Query-Time Index for Top-k Document Retrieval

  • Conference paper
Combinatorial Pattern Matching (CPM 2012)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 7354))

Included in the following conference series:

Abstract

Let \(\cal{D} = \) {d 1,d 2,...d D } be a given set of D string documents of total length n, our task is to index \(\cal{D}\), such that the k most relevant documents for an online query pattern P of length p can be retrieved efficiently. We propose an index of size |CSA| + nlogD(2 + o(1)) bits and O(t s (p) + kloglogn + polyloglogn) query time for the basic relevance metric term-frequency, where |CSA| is the size (in bits) of a compressed full text index of \(\cal{D}\), with O(t s (p)) time for searching a pattern of length p. We further reduce the space to |CSA| + nlogD(1 + o(1)) bits, however the query time will be O(t s (p) + k(logσloglogn)1 + ε + polyloglogn), where σ is the alphabet size and ε > 0 is any constant.

This work is supported in part by Taiwan NSC Grant 99-2221-E-007-123 (W. Hon) and US NSF Grant CCF–1017623 (R. Shah).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Belazzougui, D., Navarro, G.: Alphabet-Independent Compressed Text Indexing. In: Demetrescu, C., Halldórsson, M.M. (eds.) ESA 2011. LNCS, vol. 6942, pp. 748–759. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  2. Belazzougui, D., Navarro, G.: Improved Compressed Indexes for Full-Text Document Retrieval. In: Grossi, R., Sebastiani, F., Silvestri, F. (eds.) SPIRE 2011. LNCS, vol. 7024, pp. 386–397. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  3. Blum, M., Floyd, R.W., Pratt, V., Rivest, R., Tarjan, R.: Time Bounds for Selection. Journal of Computer and System Sciences 7(4), 448–481 (1973)

    Article  MathSciNet  MATH  Google Scholar 

  4. Shane Culpepper, J., Navarro, G., Puglisi, S.J., Turpin, A.: Top-k Ranked Document Search in General Text Databases. In: de Berg, M., Meyer, U. (eds.) ESA 2010. LNCS, vol. 6347, pp. 194–205. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  5. Ferragina, P., Manzini, G., Mäkinen, V., Navarro, G.: Compressed representations of sequences and full-text indexes. ACM Trans. Alg. 3(2), art. 20 (2007)

    Google Scholar 

  6. Fischer, J.: Optimal Succinctness for Range Minimum Queries. In: López-Ortiz, A. (ed.) LATIN 2010. LNCS, vol. 6034, pp. 158–169. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  7. Frederickson, G.N.: An Optimal Algorithm for Selection in a Min-Heap. Information and Computation 104(2), 197–214 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  8. Gagie, T., Navarro, G., Puglisi, S.J.: Colored Range Queries and Document Retrieval. In: Chavez, E., Lonardi, S. (eds.) SPIRE 2010. LNCS, vol. 6393, pp. 67–81. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  9. Golynski, A., Munro, J.I., Rao, S.S.: Rank/Select Operations on Large Alphabets: A Tool for Text Indexing. In: SODA, pp. 368–373 (2006)

    Google Scholar 

  10. Grossi, R., Vitter, J.S.: Compressed Suffix Arrays and Suffix Trees with Applications to Text Indexing and String Matching. SIAM Journal on Computing 35(2), 378–407 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  11. Grossi, R., Gupta, A., Vitter, J.S.: High-Order Entropy-Compressed Text Indexes. In: SODA, pp. 841–850 (2003)

    Google Scholar 

  12. Hon, W.K., Patil, M., Shah, R., Wu, S.-B.: Efficient Index for Retrieving Top-k Most Frequent Documents. Journal of Discrete Algorithms 8(4), 402–417 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  13. Hon, W.-K., Shah, R., Thankachan, S.V., Vitter, J.S.: String Retrieval for Multi-pattern Queries. In: Chavez, E., Lonardi, S. (eds.) SPIRE 2010. LNCS, vol. 6393, pp. 55–66. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  14. Hon, W.-K., Shah, R., Thankachan, S.V.: Towards an Optimal Space-and-Query-Time Index for Top-k Document Retrieval. In: Kärkkäinen, J., Stoye, J. (eds.) CPM 2012. LNCS, vol. 7354, pp. 173–184. Springer, Heidelberg (2012)

    Google Scholar 

  15. Hon, W.K., Shah, R., Vitter, J.S.: Space-Efficient Framework for Top-k String Retrieval Problems. In: FOCS, pp. 713–722 (2009)

    Google Scholar 

  16. Hon, W.-K., Shah, R., Vitter, J.S.: Compression, Indexing, and Retrieval for Massive String Data. In: Amir, A., Parida, L. (eds.) CPM 2010. LNCS, vol. 6129, pp. 260–274. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  17. Jansson, J., Sadakane, K., Sung, W.K.: Ultra-succinct Representation of Ordered Trees. In: SODA, pp. 575–584 (2007)

    Google Scholar 

  18. Karpinski, M., Nekrich, Y.: Top-k Color Queries for Document Retrieval. In: SODA, pp. 401–411 (2011)

    Google Scholar 

  19. Manber, U., Myers, G.: Suffix Arrays: A New Method for On-Line String Searches. SIAM Journal on Computing 22(5), 935–948 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  20. Matias, Y., Muthukrishnan, S.M., Şahinalp, S.C., Ziv, J.: Augmenting Suffix Trees, with Applications. In: Bilardi, G., Pietracaprina, A., Italiano, G.F., Pucci, G. (eds.) ESA 1998. LNCS, vol. 1461, pp. 67–78. Springer, Heidelberg (1998)

    Google Scholar 

  21. McCreight, E.M.: A Space-Economical Suffix Tree Construction Algorithm. Journal of the ACM 23(2), 262–272 (1976)

    Article  MathSciNet  MATH  Google Scholar 

  22. Muthukrishnan, S.: Efficient Algorithms for Document Retrieval Problems. In: SODA, pp. 657–666 (2002)

    Google Scholar 

  23. Navarro, G., Nekrich, Y.: Top-k document retrieval in optimal time and linear space. In: SODA, pp. 1066–1077 (2012)

    Google Scholar 

  24. Navarro, G., Puglisi, S.J., Valenzuela, D.: Practical Compressed Document Retrieval. In: Pardalos, P.M., Rebennack, S. (eds.) SEA 2011. LNCS, vol. 6630, pp. 193–205. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  25. Navarro, G., Valenzuela, D.: Space-Efficient Top-k Document Retrieval. To appear in SEA (2012)

    Google Scholar 

  26. Navarro, G., Puglisi, S.J.: Dual-Sorted Inverted Lists. In: Chavez, E., Lonardi, S. (eds.) SPIRE 2010. LNCS, vol. 6393, pp. 309–321. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  27. Patil, M., Thankachan, S.V., Shah, R., Hon, W.K., Vitter, J.S., Chandrasekaran, S.: Inverted Indexes for Phrases and Strings. In: SIGIR, pp. 555–564 (2011)

    Google Scholar 

  28. Raman, R., Raman, V., Rao, S.: Succinct Indexable Dictionaries with Applications to Encoding k-ary Trees, Prefix Sums and Multisets. ACM Transactions on Algorithms 3(4) (2007)

    Google Scholar 

  29. Sadakane, K.: Succinct Data Structures for Flexible Text Retrieval Systems. Journal of Discrete Algorithms 5(1), 12–22 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  30. Välimäki, N., Mäkinen, V.: Space-Efficient Algorithms for Document Retrieval. In: Ma, B., Zhang, K. (eds.) CPM 2007. LNCS, vol. 4580, pp. 205–215. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  31. Weiner, P.: Linear Pattern Matching Algorithms. In: SWAT (1973)

    Google Scholar 

  32. Willard, D.E.: Log-logarithmic Worst-Case Range Queries Are Possible in Space Θ(N). Information Processing Letters 17(2), 81–84 (1983)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of CS, National Tsing Hua University, Taiwan

    Wing-Kai Hon

  2. Department of CS, Louisiana State University, USA

    Rahul Shah & Sharma V. Thankachan

Authors
  1. Wing-Kai Hon
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rahul Shah
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sharma V. Thankachan
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Computer Science, University of Helsinki, Gustaf Hällström Katu 2b, P.O. Box 68, 00014, Helsinki, Finland

    Juha Kärkkäinen

  2. Faculty of Technology, University of Bielefeld, Universitätsstraße 25, 33615, Bielefeld, Germany

    Jens Stoye

Rights and permissions

Reprints and permissions

Copyright information

© 2012 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Hon, WK., Shah, R., Thankachan, S.V. (2012). Towards an Optimal Space-and-Query-Time Index for Top-k Document Retrieval. In: Kärkkäinen, J., Stoye, J. (eds) Combinatorial Pattern Matching. CPM 2012. Lecture Notes in Computer Science, vol 7354. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-31265-6_14

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-31265-6_14

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-31264-9

  • Online ISBN: 978-3-642-31265-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation