Skip to main content

Ranked Document Retrieval in (Almost) No Space

  • Conference paper

Part of the Lecture Notes in Computer Science book series (LNTCS,volume 7608)

Abstract

Ranked document retrieval is a fundamental task in search engines. Such queries are solved with inverted indexes that require additional 45%-80% of the compressed text space, and take tens to hundreds of microseconds per query. In this paper we show how ranked document retrieval queries can be solved within tens of milliseconds using essentially no extra space over an in-memory compressed representation of the document collection. More precisely, we enhance wavelet trees on bytecodes (WTBCs), a data structure that rearranges the bytes of the compressed collection, so that they support ranked conjunctive and disjunctive queries, using just 6%–18% of the compressed text space.

Partially funded by MICINN (PGE and FEDER) grant TIN2009-14560-C03-02, and by Xunta de Galicia (co-funded with FEDER) ref. 2010/17, for the Spanish group; by MICINN FPU program, ref. AP2007-02484 for the second author; and by Fondecyt grant 1-110066, Chile, for the third author.

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (Canada)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (Canada)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (Canada)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Zobel, J., Moffat, A.: Inverted files for text search engines. ACM Comp. Surv. 38(2) (2006)

    Google Scholar 

  2. Baeza-Yates, R., Ribeiro-Neto, B.: Modern Information Retrieval, 2nd edn. Addison-Wesley (2011)

    Google Scholar 

  3. Croft, B., Metzler, D., Strohman, T.: Search Engines: Information Retrieval in Practice. Pearson Education (2009)

    Google Scholar 

  4. Strohman, T., Croft, B.: Efficient document retrieval in main memory. In: Proc. 30th SIGIR, pp. 175–182 (2007)

    Google Scholar 

  5. Transier, F., Sanders, P.: Engineering basic algorithms of an in-memory text search engine. ACM Trans. Inf. Sys. 29(1), 2:1–2:37 (2010)

    Google Scholar 

  6. Culpepper, S., Moffat, A.: Efficient set intersection for inverted indexing. ACM Trans. Inf. Sys. 29(1) (2010)

    Google Scholar 

  7. Witten, I., Moffat, A., Bell, T.: Managing Gigabytes, 2nd edn. Morgan Kaufmann Publishers (1999)

    Google Scholar 

  8. Baeza-Yates, R., Moffat, A., Navarro, G.: Searching large text collections. In: Handbook of Massive Data Sets, pp. 195–244. Kluwer Academic Publishers (2002)

    Google Scholar 

  9. Brisaboa, N., Fariña, A., Ladra, S., Navarro, G.: Implicit indexing of natural language text by reorganizing bytecodes. Inf. Retr. (2012) (av. online)

    Google Scholar 

  10. Arroyuelo, D., González, S., Oyarzún, M.: Compressed Self-indices Supporting Conjunctive Queries on Document Collections. In: Chavez, E., Lonardi, S. (eds.) SPIRE 2010. LNCS, vol. 6393, pp. 43–54. Springer, Heidelberg (2010)

    CrossRef  Google Scholar 

  11. Brisaboa, N., Fariña, A., Navarro, G., Paramá, J.: Lightweight natural language text compression. Inf. Retr. 10(1), 1–33 (2007)

    CrossRef  Google Scholar 

  12. Heaps, H.: Information Retrieval - Computational and Theoretical Aspects. Academic Press (1978)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2012 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Brisaboa, N.R., Cerdeira-Pena, A., Navarro, G., Pedreira, Ó. (2012). Ranked Document Retrieval in (Almost) No Space. In: Calderón-Benavides, L., González-Caro, C., Chávez, E., Ziviani, N. (eds) String Processing and Information Retrieval. SPIRE 2012. Lecture Notes in Computer Science, vol 7608. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-34109-0_16

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-34109-0_16

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-34108-3

  • Online ISBN: 978-3-642-34109-0

  • eBook Packages: Computer ScienceComputer Science (R0)