Query Performance Prediction: Evaluation Contrasted with Effectiveness

  • Claudia Hauff
  • Leif Azzopardi
  • Djoerd Hiemstra
  • Franciska de Jong
Conference paper

DOI: 10.1007/978-3-642-12275-0_20

Part of the Lecture Notes in Computer Science book series (LNCS, volume 5993)
Cite this paper as:
Hauff C., Azzopardi L., Hiemstra D., de Jong F. (2010) Query Performance Prediction: Evaluation Contrasted with Effectiveness. In: Gurrin C. et al. (eds) Advances in Information Retrieval. ECIR 2010. Lecture Notes in Computer Science, vol 5993. Springer, Berlin, Heidelberg

Abstract

Query performance predictors are commonly evaluated by reporting correlation coefficients to denote how well the methods perform at predicting the retrieval performance of a set of queries. Despite the amount of research dedicated to this area, one aspect remains neglected: how strong does the correlation need to be in order to realize an improvement in retrieval effectiveness in an operational setting? We address this issue in the context of two settings: Selective Query Expansion and Meta-Search. In an empirical study, we control the quality of a predictor in order to examine how the strength of the correlation achieved, affects the effectiveness of an adaptive retrieval system. The results of this study show that many existing predictors fail to achieve a correlation strong enough to reliably improve the retrieval effectiveness in the Selective Query Expansion as well as the Meta-Search setting.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Claudia Hauff
    • 1
  • Leif Azzopardi
    • 2
  • Djoerd Hiemstra
    • 1
  • Franciska de Jong
    • 1
  1. 1.University of TwenteEnschedeThe Netherlands
  2. 2.University of GlasgowGlasgowUK

Personalised recommendations