Advertisement

Ranked Document Retrieval in (Almost) No Space

  • Nieves R. Brisaboa
  • Ana Cerdeira-Pena
  • Gonzalo Navarro
  • Óscar Pedreira
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7608)

Abstract

Ranked document retrieval is a fundamental task in search engines. Such queries are solved with inverted indexes that require additional 45%-80% of the compressed text space, and take tens to hundreds of microseconds per query. In this paper we show how ranked document retrieval queries can be solved within tens of milliseconds using essentially no extra space over an in-memory compressed representation of the document collection. More precisely, we enhance wavelet trees on bytecodes (WTBCs), a data structure that rearranges the bytes of the compressed collection, so that they support ranked conjunctive and disjunctive queries, using just 6%–18% of the compressed text space.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Zobel, J., Moffat, A.: Inverted files for text search engines. ACM Comp. Surv. 38(2) (2006)Google Scholar
  2. 2.
    Baeza-Yates, R., Ribeiro-Neto, B.: Modern Information Retrieval, 2nd edn. Addison-Wesley (2011)Google Scholar
  3. 3.
    Croft, B., Metzler, D., Strohman, T.: Search Engines: Information Retrieval in Practice. Pearson Education (2009)Google Scholar
  4. 4.
    Strohman, T., Croft, B.: Efficient document retrieval in main memory. In: Proc. 30th SIGIR, pp. 175–182 (2007)Google Scholar
  5. 5.
    Transier, F., Sanders, P.: Engineering basic algorithms of an in-memory text search engine. ACM Trans. Inf. Sys. 29(1), 2:1–2:37 (2010)Google Scholar
  6. 6.
    Culpepper, S., Moffat, A.: Efficient set intersection for inverted indexing. ACM Trans. Inf. Sys. 29(1) (2010)Google Scholar
  7. 7.
    Witten, I., Moffat, A., Bell, T.: Managing Gigabytes, 2nd edn. Morgan Kaufmann Publishers (1999)Google Scholar
  8. 8.
    Baeza-Yates, R., Moffat, A., Navarro, G.: Searching large text collections. In: Handbook of Massive Data Sets, pp. 195–244. Kluwer Academic Publishers (2002)Google Scholar
  9. 9.
    Brisaboa, N., Fariña, A., Ladra, S., Navarro, G.: Implicit indexing of natural language text by reorganizing bytecodes. Inf. Retr. (2012) (av. online)Google Scholar
  10. 10.
    Arroyuelo, D., González, S., Oyarzún, M.: Compressed Self-indices Supporting Conjunctive Queries on Document Collections. In: Chavez, E., Lonardi, S. (eds.) SPIRE 2010. LNCS, vol. 6393, pp. 43–54. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  11. 11.
    Brisaboa, N., Fariña, A., Navarro, G., Paramá, J.: Lightweight natural language text compression. Inf. Retr. 10(1), 1–33 (2007)CrossRefGoogle Scholar
  12. 12.
    Heaps, H.: Information Retrieval - Computational and Theoretical Aspects. Academic Press (1978)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Nieves R. Brisaboa
    • 1
  • Ana Cerdeira-Pena
    • 1
  • Gonzalo Navarro
    • 2
  • Óscar Pedreira
    • 1
  1. 1.Database Lab.Univ. of A CoruñaSpain
  2. 2.Dept. of Computer ScienceUniv. of ChileChile

Personalised recommendations