Column Stores as an IR Prototyping Tool

  • Hannes Mühleisen
  • Thaer Samar
  • Jimmy Lin
  • Arjen P. de Vries
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8416)


We make the suggestion that instead of implementing custom index structures and query evaluation algorithms, IR researchers should simply store document representations in a column-oriented relational database and write ranking models using SQL. For rapid prototyping, this is particularly advantageous since researchers can explore new ranking functions and features by simply issuing SQL queries, without needing to write imperative code. We demonstrate the feasibility of this approach by an implementation of conjunctive BM25 using MonetDB on a part of the ClueWeb12 collection.


Relational Database Ranking Function Query Term Query Evaluation Document Ranking 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Asadi, N., Lin, J.: Effectiveness/efficiency tradeoffs for candidate generation in multi-stage retrieval architectures. In: SIGIR, pp. 997–1000 (2013)Google Scholar
  2. 2.
    Copeland, G., Khoshafian, S.: A decomposition storage model. In: SIGMOD (1985)Google Scholar
  3. 3.
    Idreos, S., Groffen, F., Nes, N., Manegold, S., Mullender, K.S., Kersten, M.L.: MonetDB: Two decades of research in column-oriented database architectures. IEEE Data Engineering Bulletin 35(1), 40–45 (2012)Google Scholar
  4. 4.
    Metzler, D., Croft, W.B.: Combining the language model and inference network approaches to retrieval. IP&M 40(5), 735–750 (2004)Google Scholar
  5. 5.
    Ounis, I., Amati, G., Plachouras, V., He, B., Macdonald, C., Lioma, C.: Terrier: A high performance and scalable information retrieval platform. In: SIGIR Workshop on Open Source Information Retrieval (2006)Google Scholar
  6. 6.
    Zobel, J., Moffat, A.: Inverted files for text search engines. ACM Computing Surveys 38(6), 1–56 (2006)Google Scholar
  7. 7.
    Zukowski, M., Heman, S., Nes, N., Boncz, P.: Super-scalar RAM-CPU cache compression. In: ICDE, pp. 59–59 (2006)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Hannes Mühleisen
    • 1
  • Thaer Samar
    • 1
  • Jimmy Lin
    • 2
  • Arjen P. de Vries
    • 1
  1. 1.Centrum Wiskunde & InformaticaAmsterdamThe Netherlands
  2. 2.University of MarylandCollege ParkUSA

Personalised recommendations